-- Logs begin at Fri 2022-05-13 21:10:11 UTC, end at Fri 2022-05-13 22:15:17 UTC. -- May 13 21:37:05.916943 dv-cinder-840516-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 13 21:37:05.924169 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 13 21:37:05.926190 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri May 13 21:37:05 2022] *** May 13 21:37:05.926190 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 13 21:37:05.926190 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 13 21:37:05.926190 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: nodename: dv-cinder-840516-1-iscsi May 13 21:37:05.926190 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: machine: x86_64 May 13 21:37:05.926190 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: clock source: unix May 13 21:37:05.926190 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: pcre jit disabled May 13 21:37:05.926190 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: detected number of CPU cores: 4 May 13 21:37:05.926695 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: current working directory: / May 13 21:37:05.926695 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: detected binary path: /usr/bin/uwsgi-core May 13 21:37:05.926695 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: your processes number limit is 47680 May 13 21:37:05.926695 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: your memory page size is 4096 bytes May 13 21:37:05.926695 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: detected max file descriptor number: 2048 May 13 21:37:05.926695 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: lock engine: pthread robust mutexes May 13 21:37:05.926695 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: thunder lock: enabled May 13 21:37:05.926695 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 13 21:37:05.926695 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 13 21:37:05.973089 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: Python main interpreter initialized at 0x55b1a9a3f9c0 May 13 21:37:05.973089 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: python threads support enabled May 13 21:37:05.973089 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: your server socket listen backlog is limited to 100 connections May 13 21:37:05.973089 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: your mercy for graceful operations on workers is 90 seconds May 13 21:37:05.973472 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: mapped 403077 bytes (393 KB) for 2 cores May 13 21:37:05.973549 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: *** Operational MODE: preforking *** May 13 21:37:05.973816 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: *** uWSGI is running in multiple interpreter mode *** May 13 21:37:05.973816 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: spawned uWSGI master process (pid: 91418) May 13 21:37:05.974026 dv-cinder-840516-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 13 21:37:05.974615 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: spawned uWSGI worker 1 (pid: 91430, cores: 1) May 13 21:37:05.975924 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: spawned uWSGI worker 2 (pid: 91431, cores: 1) May 13 21:37:05.976167 dv-cinder-840516-1-iscsi devstack@keystone.service[91418]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 13 21:37:11.736848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 13 21:37:11.736848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: from cryptography.utils import int_from_bytes May 13 21:37:11.738736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 13 21:37:11.738736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: from cryptography.utils import int_from_bytes May 13 21:37:11.739339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 13 21:37:11.739339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: from cryptography.utils import int_from_bytes May 13 21:37:11.740103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 13 21:37:11.740103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: from cryptography.utils import int_from_bytes May 13 21:37:11.788659 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 13 21:37:11.788927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 13 21:37:11.789070 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] command line args: [] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 13 21:37:11.789292 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 13 21:37:11.789385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 13 21:37:11.789455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ================================================================================ {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 13 21:37:11.789520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 13 21:37:11.789688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] command line args: [] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 13 21:37:11.789768 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] admin_token = **** {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.789938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 13 21:37:11.790026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] config_dir = [] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.790164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ================================================================================ {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 13 21:37:11.790243 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.790368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] config_source = [] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.790443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] admin_token = **** {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.790508 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] debug = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.790649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] config_dir = [] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.790830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.790921 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.791161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] config_source = [] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.791161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] debug = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.791368 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.792333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.792333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.792333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.792333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.792569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] list_limit = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.792634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] log_config_append = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.792828 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.792915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.792982 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] log_dir = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.793120 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.793209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] log_file = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.793386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.793573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.793715 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.793871 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.794004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.794142 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.794279 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.794437 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] list_limit = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.794581 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.794742 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.794875 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] log_config_append = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.795012 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.795139 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.795270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.795401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] log_dir = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.795533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.795664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.795801 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] log_file = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.795935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.796067 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.796193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.796329 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.796488 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.796643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.796896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.797035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.35/identity {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.797211 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] publish_errors = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.797354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.797490 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.797617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.797761 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.797929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.797929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.798194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.798320 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.798444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.798576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.798734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.798883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] standard_threads = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.799016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.799146 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.799277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.799402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.799533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.799669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.799669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] use_journal = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.799912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.800046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] use_json = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.800229 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.800371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] use_stderr = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.800371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] use_syslog = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.800620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.800754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.800754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.800990 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.35/identity {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.801131 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.801267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] publish_errors = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.801392 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.801517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.801650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.801787 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.801787 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.802029 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.802154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.802279 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.802412 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.802543 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.802668 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] standard_threads = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.802819 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.802947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.803072 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.803208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.803347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.803474 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.803608 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.803734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] use_journal = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.803859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.803996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] use_json = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.803996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] use_stderr = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.804244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.804375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] use_syslog = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.804506 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.804705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 21:37:11.804834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.804966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.805092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.805217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.805357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.805491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.805623 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.805748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.805872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.805997 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.806127 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.806271 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.806398 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.806533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.806664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.806810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.806937 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.807077 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.807213 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.807338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.807463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.807589 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.807719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.807852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.807977 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.808103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.808234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.808360 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.808485 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.808618 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.808744 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.808883 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.809015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.809142 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.809268 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.809400 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.809527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.809652 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.809777 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.809901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.810026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.810162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.810294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.810425 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.810568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.810694 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.810852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.811051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.811189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.811315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.811446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.811572 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.811696 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.811828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.811954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.812078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.812202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.812327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.812473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.812599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.812724 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.812848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.812979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.813104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.813240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.813372 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.813543 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.813679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.813805 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.813936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.814072 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.815313 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.815494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.815721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.815888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.816113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.816319 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.816488 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.816677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.816867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.817054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.817249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.817479 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.817641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.817828 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.818034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.818224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.818418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.818643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.818905 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.819155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.819353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.819353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.819693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.819919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.820081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.820269 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.820430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.820613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.820748 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.820887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.821045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.821182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.821309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.821435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.821561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.821726 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.821868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.821997 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.822131 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.822259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.822385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.822520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.822658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.822816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.822816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.823090 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.823221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.823347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.823481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.823614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.823747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.823874 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.824000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.824138 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.824275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.824415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.824542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.824669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.824795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.824921 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.825056 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.825192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.825325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.825452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.825584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.825742 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.825907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.826057 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.826057 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.826304 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.826430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.826557 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.826682 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.826907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.827039 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.827167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.827294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.827426 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.827564 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.827691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.827817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.827953 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.828087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.828213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.828347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.828481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.828607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.828739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.828866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.828992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.829656 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.829789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.829915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.830042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.830170 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.830306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.830440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.830577 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.830801 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.830946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.831074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.831203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.831328 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.831454 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.831594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.831727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.831853 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.831980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.832105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.832242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.832377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] auth.external = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.832505 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.832632 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.832758 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.832758 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.832996 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.833131 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.833260 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.833387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.833518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] auth.password = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.833645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.833770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] auth.token = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.833916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.834044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.834171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.834297 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.834429 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.834564 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.834745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.834888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.835025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.835152 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.835296 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.835520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.835681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.835812 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.835959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] auth.external = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.836088 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] credential.caching = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.836215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.836342 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.836470 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.836606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.836735 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.836862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.836989 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] auth.password = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.837114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.837241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] auth.token = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.837376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.837514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.837651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.837779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.837906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.838033 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.838169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.838304 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.838431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.838560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.838687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.838836 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.838974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.839103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.839230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.839358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] credential.caching = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.839358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.839608 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.839749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.839878 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.840013 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.840141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.840268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.840394 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] federation.caching = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.840533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.840663 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.840790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.840916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.841042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.841184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.841320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.841448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.841581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.841714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.841842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.841968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.842103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.842231 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.842359 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.842493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.842621 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.842794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.843002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.843142 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.843276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.843404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.843531 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] federation.caching = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.843658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity.caching = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.843798 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.843926 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.844053 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.844179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.844306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.844431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.844566 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.844716 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.845384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.845515 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.845643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.845770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.845906 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.846036 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.846162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.846289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.846416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.846543 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.846678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.846828 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.846957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.847085 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.847221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity.caching = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.847348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.847486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.847614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.847741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.847868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.847995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.848130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.848258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.848385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.848513 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.848639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.848766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.848914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.849044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.849171 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.849299 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.849426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.849553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.849687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.849816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.849943 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.850071 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.850197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.850323 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.850457 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.850586 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.850743 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.850878 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.851005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.851133 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.851270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.851398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.851526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.851660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.851789 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.851922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.852058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.852187 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.852314 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.852450 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.852577 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.852704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.852838 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.852967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.853093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.853221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.853347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.853473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.853610 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.853738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.853865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.853992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.854118 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.854255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.854390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.854518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.854645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.854808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.854937 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.855064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.855201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.855331 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.855458 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.855585 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.855711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.855837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.855971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.856202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.856365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.856496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.856624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.856751 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.856886 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.857015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.857142 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.857277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.857412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.857540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.857675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.857803 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.857930 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.858067 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.858195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.858322 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.858458 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.858586 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.858729 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.858868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.858995 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.859260 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.859389 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.859518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.859645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.859781 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.859908 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.860043 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.860171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.860298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.860451 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.860588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.860716 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.860851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.860979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.861106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.861233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.861360 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.861494 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.861494 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.861494 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.861868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.862002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.862130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.862265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.862393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.862520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.862647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.862828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.862958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.863097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.863226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.863364 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.863491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.863619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.863747 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.863886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.864014 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.864141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.864274 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.864402 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.864528 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.864662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.864791 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.864919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.865046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.865182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.865310 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.865445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.865574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.865701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.865830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.865957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.866084 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.866218 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.866346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.866473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.866599 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.866742 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.866876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.867022 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.867151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.867278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.867411 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.867539 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.867665 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] resource.caching = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.867800 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.867929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.868055 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.868182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.868309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.868435 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.868631 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.868770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.868899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.869027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.869155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.869282 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.869416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.869544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.869671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.869797 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.869924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.870050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.870185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] resource.caching = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.870313 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] role.caching = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.870439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.870575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] role.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.870717 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.870851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.870990 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.871119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.871246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.871373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.871501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.871633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.871773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.871908 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.872036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.872164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.872291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.872427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.872563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.872691 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.872857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] role.caching = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.872987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.873115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] role.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.873242 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.873377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.873505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.873632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.873758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.873886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.874013 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.874163 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.874292 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.874420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.874420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.874659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.874850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.874981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.875109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.875237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.875365 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.875491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.875626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.875822 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.875997 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.876132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.876262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.876389 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.876526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.876655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.876782 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.876909 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.877036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.877163 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.877298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.877427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.877554 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.877692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.877820 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.877947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.878083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.878211 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.878340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.878467 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.878924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.879062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.879201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.879329 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.879457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.879584 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.879710 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.879849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.879985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.880114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] token.caching = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.880241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.880367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.880495 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.880622 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.880756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.880884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.881010 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.881143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.881272 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.881406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.881543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.881679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.881807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.881933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.882060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.882198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.882326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] token.caching = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.882453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.882581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.882725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.882858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.882996 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.883216 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.883377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.883518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.883647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.883774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.883909 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.884038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.884165 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.884291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.884418 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.884545 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.884681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.884810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.884937 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.connection = **** {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.885072 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.885208 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.885346 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.885474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.885601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.885736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.885865 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.885991 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.886117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.886117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.886356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.886356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.886605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.886780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.886915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.887051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.887180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.connection = **** {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.887322 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.887462 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.887591 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.887718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.887846 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.887974 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.888100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.888235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.888364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.888491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.888618 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.888745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.888882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.889687 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.890929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.893615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.894650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.894887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.895034 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.895257 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.895400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 13 21:37:11.895540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.895540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.895540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.895540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.895540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.895540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.895540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.895540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 21:37:11.895540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 13 21:37:11.908239 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:11.908471 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:11.908693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:11.909599 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:11.909835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:11.910067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:11.919349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:11.919578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:11.951817 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:11.952103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:11.953439 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:11.953682 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.005417 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.006428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.060666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.060949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.068083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.068329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.129965 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.130252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.130401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.130617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.130864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.131017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.131162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.131302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.131302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.131623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.131842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.132055 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.155701 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.155906 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.156062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.156210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.156354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.156620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.156784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.156940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.157096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.157237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.157379 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.157518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.157667 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.157808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.168767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.170399 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.183550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.183824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.183994 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.184214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.184436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.184584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.184725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.184873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.185014 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.185154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.185293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.185484 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.185697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.185908 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 21:37:12.203068 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.203336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 13 21:37:12.203605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.203813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 13 21:37:12.203968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.204716 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 13 21:37:12.204871 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.204871 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.205138 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.205138 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 13 21:37:12.205434 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.205575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.205714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.205851 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.205851 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.206124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.216456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.216741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.217012 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.217293 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.218773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.219064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.219331 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.219696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.225670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.228979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.229260 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.229542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.229942 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.230228 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.230581 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.230957 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.231417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.231898 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.232364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.232820 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.235448 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.241605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.241981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.242431 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.243046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.243468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.243861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.244262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.244674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.245075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.245486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.245883 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.260583 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.262187 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.262467 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.268457 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.268744 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.269026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.276449 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.276695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.281359 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.281647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.281922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.282203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.282533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.282831 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.283105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.283371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.294121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.296989 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.297424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.302114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.302407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.306983 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.307163 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.307464 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.307910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.310602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.310921 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.316449 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.320012 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.320141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.320297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.323787 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.324081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.324367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.324650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.324923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.324998 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.325198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.325270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.325489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.325768 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.326037 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.326305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.326570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.326856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.344816 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.345197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.345465 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.345726 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.345928 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.346204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.350678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.353592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.354265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.354553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.355605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.355888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.358971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.359250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.359484 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.359576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.359796 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.360074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.363159 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.366873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.367168 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.367444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.367720 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.367993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.368264 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.373537 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.373819 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.374096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.374369 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.374642 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.375726 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.375726 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.387930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.388307 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.388574 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.388833 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.394740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.397241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.397808 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.398966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.399233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.401139 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.402249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.402526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.402760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.402913 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.403083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.403353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.404388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.404664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.404957 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.412427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.414030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.416676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.416952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.417226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.417377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.417518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.417670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.417810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.417947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.418092 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.418229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.418376 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.427282 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.427575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.427857 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.435816 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.436079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.436232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.436372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.436606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.436870 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.437152 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.437420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.437698 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.437964 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.438230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.439030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.439314 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.439575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.439847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.439990 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.443268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.444856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.446470 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.446759 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.447048 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.455180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.456746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.459959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.460244 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.460524 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.460799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.469981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.470269 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.470545 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.479218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55b1a9a3f9c0 pid: 91431 (default app) May 13 21:37:12.479434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 21:37:12.479682 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.479881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.480149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.480415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.480698 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.480965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.481240 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.481501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.481761 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.482025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.482295 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.482554 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.482735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f46fa-e9f2-4ee3-a97a-12d46b7be6ad None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:12.482887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 21:37:12.483028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f46fa-e9f2-4ee3-a97a-12d46b7be6ad None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:12.483226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f46fa-e9f2-4ee3-a97a-12d46b7be6ad None None] PATH_INFO: `/v3` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:12.489726 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1/1] 10.222.0.35 () {50 vars in 791 bytes} [Fri May 13 21:37:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:12.523866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55b1a9a3f9c0 pid: 91430 (default app) May 13 21:37:20.442621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09c5c5-ab8a-4c2e-bc4c-b59c5c7a6d8a None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:20.443197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09c5c5-ab8a-4c2e-bc4c-b59c5c7a6d8a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:20.443298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09c5c5-ab8a-4c2e-bc4c-b59c5c7a6d8a None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:20.444601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 2/2] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:20.452345 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7891f05-8be8-4a57-8d19-5d80fbf774cd None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:20.452775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7891f05-8be8-4a57-8d19-5d80fbf774cd None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:20.452922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7891f05-8be8-4a57-8d19-5d80fbf774cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:20.483615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG oslo_db.sqlalchemy.engines [None req-b7891f05-8be8-4a57-8d19-5d80fbf774cd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91430) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 13 21:37:20.672082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG passlib.handlers.bcrypt [None req-b7891f05-8be8-4a57-8d19-5d80fbf774cd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91430) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 13 21:37:20.672499 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG passlib.handlers.bcrypt [None req-b7891f05-8be8-4a57-8d19-5d80fbf774cd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91430) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 13 21:37:20.729887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-b7891f05-8be8-4a57-8d19-5d80fbf774cd None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:20.731449 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b7891f05-8be8-4a57-8d19-5d80fbf774cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:20.856095 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1/3] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 13 21:37:21.010415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09c5c5-ab8a-4c2e-bc4c-b59c5c7a6d8a None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:21.043239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG oslo_db.sqlalchemy.engines [None req-2c09c5c5-ab8a-4c2e-bc4c-b59c5c7a6d8a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91431) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 13 21:37:21.168486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09c5c5-ab8a-4c2e-bc4c-b59c5c7a6d8a None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:21.173322 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a127c9-a78d-4f76-8f3b-b1759a3253fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:21.173970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a127c9-a78d-4f76-8f3b-b1759a3253fa admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:21.174256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a127c9-a78d-4f76-8f3b-b1759a3253fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:21.174522 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a127c9-a78d-4f76-8f3b-b1759a3253fa admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:21.183082 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a127c9-a78d-4f76-8f3b-b1759a3253fa admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:21.243408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: WARNING keystone.common.rbac_enforcer.enforcer [None req-73a127c9-a78d-4f76-8f3b-b1759a3253fa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 13 21:37:21.243747 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a127c9-a78d-4f76-8f3b-b1759a3253fa admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:21.254034 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 3/4] 10.222.0.35 () {58 vars in 1324 bytes} [Fri May 13 21:37:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 13 21:37:21.259899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7891f05-8be8-4a57-8d19-5d80fbf774cd None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:21.273923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7891f05-8be8-4a57-8d19-5d80fbf774cd None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:21.278233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4a9ad-985c-416c-9d1d-91659117792f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:21.279363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4a9ad-985c-416c-9d1d-91659117792f admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:21.279653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4a9ad-985c-416c-9d1d-91659117792f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:21.279927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4a9ad-985c-416c-9d1d-91659117792f admin admin] PATH_INFO: `/v3/projects/44b04a795b974ef6b1191a457380d73c` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:21.281419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4a9ad-985c-416c-9d1d-91659117792f admin admin] RBAC: Authorizing `identity:get_project(project_id=44b04a795b974ef6b1191a457380d73c)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:21.333242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f6d4a9ad-985c-416c-9d1d-91659117792f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 13 21:37:21.333587 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4a9ad-985c-416c-9d1d-91659117792f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:21.338374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 2/5] 10.222.0.35 () {56 vars in 1207 bytes} [Fri May 13 21:37:21 2022] GET /identity/v3/projects/44b04a795b974ef6b1191a457380d73c => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:23.557638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc4323-fd79-4831-94ae-8d1f43d3a709 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:23.558201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc4323-fd79-4831-94ae-8d1f43d3a709 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:23.558201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc4323-fd79-4831-94ae-8d1f43d3a709 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:23.562290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 4/6] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:37:23.567780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4b034-51b1-4750-82dd-dee860ad286d None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:23.568178 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4b034-51b1-4750-82dd-dee860ad286d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:23.568344 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4b034-51b1-4750-82dd-dee860ad286d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:23.587911 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-60e4b034-51b1-4750-82dd-dee860ad286d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:23.589443 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-60e4b034-51b1-4750-82dd-dee860ad286d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:23.597769 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 3/7] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 13 21:37:23.619866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc4323-fd79-4831-94ae-8d1f43d3a709 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:23.631941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc4323-fd79-4831-94ae-8d1f43d3a709 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:23.636193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e472355-d837-4daf-b8ca-8e30d542e220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:23.636817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e472355-d837-4daf-b8ca-8e30d542e220 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:23.637104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e472355-d837-4daf-b8ca-8e30d542e220 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:23.637383 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e472355-d837-4daf-b8ca-8e30d542e220 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:23.645051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e472355-d837-4daf-b8ca-8e30d542e220 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:23.646852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e472355-d837-4daf-b8ca-8e30d542e220 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:23.655432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 5/8] 10.222.0.35 () {58 vars in 1324 bytes} [Fri May 13 21:37:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:37:23.661076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4b034-51b1-4750-82dd-dee860ad286d None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:23.669791 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4b034-51b1-4750-82dd-dee860ad286d None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:23.674718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009a10b-56f2-42d2-beba-ccbd2755dce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:23.675567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009a10b-56f2-42d2-beba-ccbd2755dce0 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:23.675848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009a10b-56f2-42d2-beba-ccbd2755dce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:23.676122 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009a10b-56f2-42d2-beba-ccbd2755dce0 admin admin] PATH_INFO: `/v3/users/3b8d43206c7f4426b0be3e6d78690460` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:23.677480 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009a10b-56f2-42d2-beba-ccbd2755dce0 admin admin] RBAC: Authorizing `identity:get_user(user_id=3b8d43206c7f4426b0be3e6d78690460)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:23.679320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009a10b-56f2-42d2-beba-ccbd2755dce0 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:23.680869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 4/9] 10.222.0.35 () {56 vars in 1198 bytes} [Fri May 13 21:37:23 2022] GET /identity/v3/users/3b8d43206c7f4426b0be3e6d78690460 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:25.963102 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a770e0-0d63-4f5c-a3c9-f29a6f04e960 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:25.963639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a770e0-0d63-4f5c-a3c9-f29a6f04e960 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:25.963639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a770e0-0d63-4f5c-a3c9-f29a6f04e960 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:25.965008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 6/10] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:25.970766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5af9ea-b799-47de-b107-430e15876240 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:25.971179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5af9ea-b799-47de-b107-430e15876240 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:25.971326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5af9ea-b799-47de-b107-430e15876240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:25.973976 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f70d4-3c75-4360-9e73-de60c3deff05 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:25.974295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f70d4-3c75-4360-9e73-de60c3deff05 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:25.974578 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f70d4-3c75-4360-9e73-de60c3deff05 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:25.975863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 7/11] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:37:25.981076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8843c55-675f-407e-b3e0-824e34ebc289 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:25.981365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8843c55-675f-407e-b3e0-824e34ebc289 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:25.981647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8843c55-675f-407e-b3e0-824e34ebc289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:26.023408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG passlib.handlers.bcrypt [None req-d8843c55-675f-407e-b3e0-824e34ebc289 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91431) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 13 21:37:26.023837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG passlib.handlers.bcrypt [None req-d8843c55-675f-407e-b3e0-824e34ebc289 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91431) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 13 21:37:26.070867 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-d8843c55-675f-407e-b3e0-824e34ebc289 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:26.072500 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d8843c55-675f-407e-b3e0-824e34ebc289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:26.096112 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-ca5af9ea-b799-47de-b107-430e15876240 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:26.097285 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ca5af9ea-b799-47de-b107-430e15876240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:26.175341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 5/12] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:26.182476 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 8/13] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:37:26.198292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5af9ea-b799-47de-b107-430e15876240 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:26.204570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8843c55-675f-407e-b3e0-824e34ebc289 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:26.297522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5af9ea-b799-47de-b107-430e15876240 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:26.300464 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8843c55-675f-407e-b3e0-824e34ebc289 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:26.329889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902eaf3-7513-4828-8b3a-a397573e6072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:26.330675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902eaf3-7513-4828-8b3a-a397573e6072 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:26.330989 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902eaf3-7513-4828-8b3a-a397573e6072 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:26.331269 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902eaf3-7513-4828-8b3a-a397573e6072 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:26.333117 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442d741-2f5a-4657-be91-6465bc8ad8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:26.334001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442d741-2f5a-4657-be91-6465bc8ad8b5 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:26.334293 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442d741-2f5a-4657-be91-6465bc8ad8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:26.334577 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442d741-2f5a-4657-be91-6465bc8ad8b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:26.342007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902eaf3-7513-4828-8b3a-a397573e6072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-6902eaf3-7513-4828-8b3a-a397573e6072 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:26.353111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:26.354293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:26.355210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:26.355210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:26.355210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:26.355210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:26.355210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:26.355210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:26.355210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:26.355210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:26.355210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:26.355210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:26.355210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:37:26.355210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:37:26.355210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 6/14] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:37:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:26.359987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902eaf3-7513-4828-8b3a-a397573e6072 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:26.401508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442d741-2f5a-4657-be91-6465bc8ad8b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:26.403055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442d741-2f5a-4657-be91-6465bc8ad8b5 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:26.452496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902eaf3-7513-4828-8b3a-a397573e6072 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:26.483884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0880b-99df-4b85-a21a-72f29e795732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:26.484524 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0880b-99df-4b85-a21a-72f29e795732 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:26.484805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0880b-99df-4b85-a21a-72f29e795732 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:26.485082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0880b-99df-4b85-a21a-72f29e795732 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:26.485831 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0880b-99df-4b85-a21a-72f29e795732 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:26.486130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0880b-99df-4b85-a21a-72f29e795732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:26.487542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0880b-99df-4b85-a21a-72f29e795732 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:26.495603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 9/15] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:37:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 21:37:26.498939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 7/16] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:26.501467 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442d741-2f5a-4657-be91-6465bc8ad8b5 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:26.504937 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0880b-99df-4b85-a21a-72f29e795732 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:26.607654 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0880b-99df-4b85-a21a-72f29e795732 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:26.643054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9f543-4cfb-4633-a7bb-18189f1b6578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:26.643917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9f543-4cfb-4633-a7bb-18189f1b6578 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:26.644201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9f543-4cfb-4633-a7bb-18189f1b6578 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:26.644477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9f543-4cfb-4633-a7bb-18189f1b6578 None admin] PATH_INFO: `/v3/users/3b8d43206c7f4426b0be3e6d78690460` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:26.645683 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9f543-4cfb-4633-a7bb-18189f1b6578 None admin] RBAC: Authorizing `identity:get_user(user_id=3b8d43206c7f4426b0be3e6d78690460)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:26.647186 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9f543-4cfb-4633-a7bb-18189f1b6578 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:26.648809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 8/17] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:26 2022] GET /identity/v3/users/3b8d43206c7f4426b0be3e6d78690460 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:26.654131 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9f543-4cfb-4633-a7bb-18189f1b6578 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:26.701879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442d741-2f5a-4657-be91-6465bc8ad8b5 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:26.734380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb325fe0-e582-4937-ae72-39759e6e1c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:26.735756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb325fe0-e582-4937-ae72-39759e6e1c56 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:26.736048 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb325fe0-e582-4937-ae72-39759e6e1c56 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:26.736329 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb325fe0-e582-4937-ae72-39759e6e1c56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:26.746119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9f543-4cfb-4633-a7bb-18189f1b6578 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:26.755410 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb325fe0-e582-4937-ae72-39759e6e1c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:26.756883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb325fe0-e582-4937-ae72-39759e6e1c56 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-cb325fe0-e582-4937-ae72-39759e6e1c56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:37:26.776671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:26.777680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:26.778824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:37:26.778824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:37:26.778824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:37:26.778824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:37:26.778824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 10/18] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:37:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:26.779612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a44c1-2101-4c81-89d5-b9dc2f76af93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:26.780390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a44c1-2101-4c81-89d5-b9dc2f76af93 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:26.780669 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a44c1-2101-4c81-89d5-b9dc2f76af93 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:26.780943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a44c1-2101-4c81-89d5-b9dc2f76af93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:26.782141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a44c1-2101-4c81-89d5-b9dc2f76af93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:26.783606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a44c1-2101-4c81-89d5-b9dc2f76af93 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:26.783717 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb325fe0-e582-4937-ae72-39759e6e1c56 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:26.785384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 9/19] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:37:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:26.790609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a44c1-2101-4c81-89d5-b9dc2f76af93 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:26.883589 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a44c1-2101-4c81-89d5-b9dc2f76af93 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:26.886636 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb325fe0-e582-4937-ae72-39759e6e1c56 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:26.915447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ecf34-af82-4ec8-9899-7a95c617d246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:26.916093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ecf34-af82-4ec8-9899-7a95c617d246 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:26.916374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ecf34-af82-4ec8-9899-7a95c617d246 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:26.916650 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ecf34-af82-4ec8-9899-7a95c617d246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:26.917477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ecf34-af82-4ec8-9899-7a95c617d246 None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.domain.id=default, user.id=3b8d43206c7f4426b0be3e6d78690460) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:26.917786 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ecf34-af82-4ec8-9899-7a95c617d246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:26.917974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe6d8b-074b-479a-bec4-53f9169db734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:26.918624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe6d8b-074b-479a-bec4-53f9169db734 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:26.918983 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe6d8b-074b-479a-bec4-53f9169db734 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:26.919259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ecf34-af82-4ec8-9899-7a95c617d246 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:26.919354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe6d8b-074b-479a-bec4-53f9169db734 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:26.919972 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe6d8b-074b-479a-bec4-53f9169db734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:26.920275 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe6d8b-074b-479a-bec4-53f9169db734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:26.921643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe6d8b-074b-479a-bec4-53f9169db734 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:26.926256 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 10/20] 10.222.0.35 () {56 vars in 1322 bytes} [Fri May 13 21:37:26 2022] GET /identity/v3/role_assignments?user.id=3b8d43206c7f4426b0be3e6d78690460&scope.domain.id=default&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 233 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:26.940490 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 11/21] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:29.710547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57ced-29e0-493f-a35f-153fa40017ca None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:29.711659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57ced-29e0-493f-a35f-153fa40017ca None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:29.711845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78243662-f014-421b-bb03-b9530a64b02e None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:29.712590 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78243662-f014-421b-bb03-b9530a64b02e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:29.713085 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57ced-29e0-493f-a35f-153fa40017ca None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:29.713563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78243662-f014-421b-bb03-b9530a64b02e None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:29.716584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 11/22] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:29.717018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 12/23] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:29.724715 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f7ab6-49da-4b0b-9c4c-4ccbc79e9ac3 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:29.726146 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be2f02-79be-4445-a089-c1240ae1a0a9 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:29.726635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be2f02-79be-4445-a089-c1240ae1a0a9 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:29.727175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f7ab6-49da-4b0b-9c4c-4ccbc79e9ac3 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:29.727672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be2f02-79be-4445-a089-c1240ae1a0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:29.728142 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f7ab6-49da-4b0b-9c4c-4ccbc79e9ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:29.759796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e2be2f02-79be-4445-a089-c1240ae1a0a9 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:29.760191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-515f7ab6-49da-4b0b-9c4c-4ccbc79e9ac3 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:29.761419 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-515f7ab6-49da-4b0b-9c4c-4ccbc79e9ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:29.761737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e2be2f02-79be-4445-a089-c1240ae1a0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:29.855259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 12/24] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:29.857578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9faed-6dca-4b24-ae3b-15b5126019db None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:29.857873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9faed-6dca-4b24-ae3b-15b5126019db None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:29.858152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9faed-6dca-4b24-ae3b-15b5126019db None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:29.859675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 13/25] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:29.865824 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967f7d5-7aed-4c36-b328-99acf43598a1 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:29.866113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967f7d5-7aed-4c36-b328-99acf43598a1 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:29.866392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967f7d5-7aed-4c36-b328-99acf43598a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:29.890054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 13/26] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:29.895782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f7ab6-49da-4b0b-9c4c-4ccbc79e9ac3 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:29.903718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-2967f7d5-7aed-4c36-b328-99acf43598a1 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:29.904913 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-2967f7d5-7aed-4c36-b328-99acf43598a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:29.994325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 14/27] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:29.995220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967f7d5-7aed-4c36-b328-99acf43598a1 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:30.119523 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967f7d5-7aed-4c36-b328-99acf43598a1 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:30.140886 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f7ab6-49da-4b0b-9c4c-4ccbc79e9ac3 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:30.167916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6996b3-a6fa-4b07-9f84-1792e43b8750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:30.168719 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6996b3-a6fa-4b07-9f84-1792e43b8750 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:30.168999 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6996b3-a6fa-4b07-9f84-1792e43b8750 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:30.169276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6996b3-a6fa-4b07-9f84-1792e43b8750 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:30.203026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6996b3-a6fa-4b07-9f84-1792e43b8750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-5b6996b3-a6fa-4b07-9f84-1792e43b8750 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:30.211914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:30.212774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:30.213517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:30.213517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:30.213517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:30.213517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:30.213517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:30.213517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:30.213517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:30.213517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:30.213517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:30.213517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:30.213517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:37:30.213517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:37:30.216187 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 15/28] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:37:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:30.217291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6996b3-a6fa-4b07-9f84-1792e43b8750 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:30.237572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cfa32-d52e-4fc1-9c38-aaf556b7a951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:30.238718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cfa32-d52e-4fc1-9c38-aaf556b7a951 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:30.239205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cfa32-d52e-4fc1-9c38-aaf556b7a951 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:30.239677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cfa32-d52e-4fc1-9c38-aaf556b7a951 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:30.251776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cfa32-d52e-4fc1-9c38-aaf556b7a951 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:30.253776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cfa32-d52e-4fc1-9c38-aaf556b7a951 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-583cfa32-d52e-4fc1-9c38-aaf556b7a951 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:37:30.307420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:30.308276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:30.309155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:37:30.309155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:37:30.309155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:37:30.309155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:37:30.309155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 14/29] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:37:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:30.310896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cfa32-d52e-4fc1-9c38-aaf556b7a951 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:30.343281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6996b3-a6fa-4b07-9f84-1792e43b8750 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:30.383187 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2029fdf-2105-44d6-9676-8e49afc9b435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:30.383879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2029fdf-2105-44d6-9676-8e49afc9b435 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:30.384164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2029fdf-2105-44d6-9676-8e49afc9b435 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:30.384438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2029fdf-2105-44d6-9676-8e49afc9b435 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:30.385204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2029fdf-2105-44d6-9676-8e49afc9b435 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:30.386673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2029fdf-2105-44d6-9676-8e49afc9b435 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:30.516567 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cfa32-d52e-4fc1-9c38-aaf556b7a951 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:30.521884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 16/30] 10.222.0.35 () {60 vars in 1131 bytes} [Fri May 13 21:37:30 2022] POST /identity/v3/roles => generated 219 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:37:30.522979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2029fdf-2105-44d6-9676-8e49afc9b435 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:30.557727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11ecaa-539e-4d8f-83ea-322ab168534f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:30.558851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11ecaa-539e-4d8f-83ea-322ab168534f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:30.559347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11ecaa-539e-4d8f-83ea-322ab168534f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:30.559859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11ecaa-539e-4d8f-83ea-322ab168534f None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:30.560854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11ecaa-539e-4d8f-83ea-322ab168534f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:30.561414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11ecaa-539e-4d8f-83ea-322ab168534f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:30.563832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11ecaa-539e-4d8f-83ea-322ab168534f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:30.579561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 15/31] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:30.581279 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6614f-61ba-48ea-9a28-080397f2b002 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:30.581817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6614f-61ba-48ea-9a28-080397f2b002 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:30.582394 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6614f-61ba-48ea-9a28-080397f2b002 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:30.584271 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 16/32] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:30.591847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510b87b-fef9-4432-9d50-4f6060c81ecf None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:30.592188 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510b87b-fef9-4432-9d50-4f6060c81ecf None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:30.592414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510b87b-fef9-4432-9d50-4f6060c81ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:30.617304 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-a510b87b-fef9-4432-9d50-4f6060c81ecf None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:30.619345 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a510b87b-fef9-4432-9d50-4f6060c81ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:30.664031 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2029fdf-2105-44d6-9676-8e49afc9b435 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:30.697018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 17/33] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:30.698323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510b87b-fef9-4432-9d50-4f6060c81ecf None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:30.720351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957561d-1a0d-4fa4-b949-0c18b4e2d5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:30.721062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957561d-1a0d-4fa4-b949-0c18b4e2d5a9 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:30.721547 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957561d-1a0d-4fa4-b949-0c18b4e2d5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:30.721997 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957561d-1a0d-4fa4-b949-0c18b4e2d5a9 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:30.723002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957561d-1a0d-4fa4-b949-0c18b4e2d5a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:30.723463 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957561d-1a0d-4fa4-b949-0c18b4e2d5a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:30.725323 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957561d-1a0d-4fa4-b949-0c18b4e2d5a9 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:30.756396 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 17/34] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:30.758146 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de62de9-8a99-43f4-9b62-d0cee2b0069a None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:30.758447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de62de9-8a99-43f4-9b62-d0cee2b0069a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:30.758751 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de62de9-8a99-43f4-9b62-d0cee2b0069a None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:30.760182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 18/35] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:30.760711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de62de9-8a99-43f4-9b62-d0cee2b0069a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:30.821588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510b87b-fef9-4432-9d50-4f6060c81ecf None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:30.869226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7920b35-ee57-48e9-a809-f7c33c879ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:30.870163 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7920b35-ee57-48e9-a809-f7c33c879ed1 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:30.870454 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7920b35-ee57-48e9-a809-f7c33c879ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:30.870743 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7920b35-ee57-48e9-a809-f7c33c879ed1 None admin] PATH_INFO: `/v3/users/3b8d43206c7f4426b0be3e6d78690460` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:30.872349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7920b35-ee57-48e9-a809-f7c33c879ed1 None admin] RBAC: Authorizing `identity:get_user(user_id=3b8d43206c7f4426b0be3e6d78690460)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:30.873822 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7920b35-ee57-48e9-a809-f7c33c879ed1 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:30.875544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 18/36] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:30 2022] GET /identity/v3/users/3b8d43206c7f4426b0be3e6d78690460 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:30.876775 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebb2ff-0b4a-45a8-92f9-bd3510823fd9 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:30.877067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebb2ff-0b4a-45a8-92f9-bd3510823fd9 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:30.877343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebb2ff-0b4a-45a8-92f9-bd3510823fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:30.892830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de62de9-8a99-43f4-9b62-d0cee2b0069a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:30.893747 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-21ebb2ff-0b4a-45a8-92f9-bd3510823fd9 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:30.894956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-21ebb2ff-0b4a-45a8-92f9-bd3510823fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:30.928836 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35cfe6-5116-4ec1-bd79-5d1aad071bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:30.929522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35cfe6-5116-4ec1-bd79-5d1aad071bcd None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:30.929812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35cfe6-5116-4ec1-bd79-5d1aad071bcd None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:30.930098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35cfe6-5116-4ec1-bd79-5d1aad071bcd None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:30.930876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35cfe6-5116-4ec1-bd79-5d1aad071bcd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:30.932419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35cfe6-5116-4ec1-bd79-5d1aad071bcd None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:30.946566 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 19/37] 10.222.0.35 () {60 vars in 1131 bytes} [Fri May 13 21:37:30 2022] POST /identity/v3/roles => generated 223 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:37:30.955784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35cfe6-5116-4ec1-bd79-5d1aad071bcd None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:30.970874 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 19/38] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:30.972051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebb2ff-0b4a-45a8-92f9-bd3510823fd9 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:31.070115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35cfe6-5116-4ec1-bd79-5d1aad071bcd None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:31.081373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebb2ff-0b4a-45a8-92f9-bd3510823fd9 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:31.107939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a5221-b692-413a-aada-389d586bf6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:31.109013 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a5221-b692-413a-aada-389d586bf6ea None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:31.109514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a5221-b692-413a-aada-389d586bf6ea None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:31.110598 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a5221-b692-413a-aada-389d586bf6ea None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:31.111639 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a5221-b692-413a-aada-389d586bf6ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:31.113514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a5221-b692-413a-aada-389d586bf6ea None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:31.119673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed93d3d-481b-401a-9643-4c421b68743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:31.120503 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed93d3d-481b-401a-9643-4c421b68743c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:31.120812 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed93d3d-481b-401a-9643-4c421b68743c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:31.121072 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed93d3d-481b-401a-9643-4c421b68743c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:31.122459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed93d3d-481b-401a-9643-4c421b68743c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:31.124070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed93d3d-481b-401a-9643-4c421b68743c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:31.126193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 20/39] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:37:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:31.127356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed93d3d-481b-401a-9643-4c421b68743c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:31.158321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 20/40] 10.222.0.35 () {60 vars in 1141 bytes} [Fri May 13 21:37:30 2022] POST /identity/v3/projects => generated 301 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:37:31.159666 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a5221-b692-413a-aada-389d586bf6ea None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:31.250592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed93d3d-481b-401a-9643-4c421b68743c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:31.293042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880419c7-a62d-45c8-b5b5-0d3d00cfc982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:31.293808 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880419c7-a62d-45c8-b5b5-0d3d00cfc982 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:31.294461 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880419c7-a62d-45c8-b5b5-0d3d00cfc982 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:31.294461 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880419c7-a62d-45c8-b5b5-0d3d00cfc982 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:31.295440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880419c7-a62d-45c8-b5b5-0d3d00cfc982 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:31.296968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880419c7-a62d-45c8-b5b5-0d3d00cfc982 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:31.317024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a5221-b692-413a-aada-389d586bf6ea None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:31.364980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fcd30-9527-4146-b713-e7f45239a8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:31.365799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fcd30-9527-4146-b713-e7f45239a8f1 None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:31.366093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fcd30-9527-4146-b713-e7f45239a8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:31.366383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fcd30-9527-4146-b713-e7f45239a8f1 None admin] PATH_INFO: `/v3/domains/default/users/3b8d43206c7f4426b0be3e6d78690460/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:31.369068 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fcd30-9527-4146-b713-e7f45239a8f1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3b8d43206c7f4426b0be3e6d78690460, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:31.370565 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fcd30-9527-4146-b713-e7f45239a8f1 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:31.390252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 21/41] 10.222.0.35 () {58 vars in 1361 bytes} [Fri May 13 21:37:31 2022] PUT /identity/v3/domains/default/users/3b8d43206c7f4426b0be3e6d78690460/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:37:31.391928 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af6195-db5b-435f-bc3f-23144baea248 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:31.392226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af6195-db5b-435f-bc3f-23144baea248 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:31.392590 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af6195-db5b-435f-bc3f-23144baea248 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:31.394079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 22/42] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:31.399628 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ba07f-38d9-4714-a60e-07ac087a22a2 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:31.399947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ba07f-38d9-4714-a60e-07ac087a22a2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:31.400265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ba07f-38d9-4714-a60e-07ac087a22a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:31.420006 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-725ba07f-38d9-4714-a60e-07ac087a22a2 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:31.421755 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-725ba07f-38d9-4714-a60e-07ac087a22a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:31.435115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 21/43] 10.222.0.35 () {60 vars in 1131 bytes} [Fri May 13 21:37:31 2022] POST /identity/v3/roles => generated 225 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:37:31.511634 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 23/44] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:37:31.535777 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880419c7-a62d-45c8-b5b5-0d3d00cfc982 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:31.647239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880419c7-a62d-45c8-b5b5-0d3d00cfc982 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:31.682685 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43ce1c-0d99-427a-ab10-0882dd7d0d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:31.683592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43ce1c-0d99-427a-ab10-0882dd7d0d5c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:31.683876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43ce1c-0d99-427a-ab10-0882dd7d0d5c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:31.684209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43ce1c-0d99-427a-ab10-0882dd7d0d5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:31.685488 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43ce1c-0d99-427a-ab10-0882dd7d0d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:31.687362 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43ce1c-0d99-427a-ab10-0882dd7d0d5c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:31.689059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 22/45] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:37:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:31.695036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ba07f-38d9-4714-a60e-07ac087a22a2 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:31.788932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ba07f-38d9-4714-a60e-07ac087a22a2 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:31.820939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d38b3-bcb9-4544-bac5-500e3657db2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:31.821598 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d38b3-bcb9-4544-bac5-500e3657db2f None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:31.821885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d38b3-bcb9-4544-bac5-500e3657db2f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:31.822165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d38b3-bcb9-4544-bac5-500e3657db2f None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:31.822920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d38b3-bcb9-4544-bac5-500e3657db2f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:31.824392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d38b3-bcb9-4544-bac5-500e3657db2f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:31.848119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 24/46] 10.222.0.35 () {60 vars in 1141 bytes} [Fri May 13 21:37:31 2022] POST /identity/v3/projects => generated 312 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:37:33.659583 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e1875-6b53-44bf-b2df-e178ef1b0846 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:33.660104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e1875-6b53-44bf-b2df-e178ef1b0846 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:33.660216 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e1875-6b53-44bf-b2df-e178ef1b0846 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:33.661535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 23/47] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:33.666756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720ee02-e640-4fde-8db6-8e7283d948aa None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:33.667155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720ee02-e640-4fde-8db6-8e7283d948aa None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:33.667332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720ee02-e640-4fde-8db6-8e7283d948aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:33.683805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-8720ee02-e640-4fde-8db6-8e7283d948aa None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:33.684992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-8720ee02-e640-4fde-8db6-8e7283d948aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:33.748282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 25/48] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:33.772201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e1875-6b53-44bf-b2df-e178ef1b0846 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:33.869616 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e1875-6b53-44bf-b2df-e178ef1b0846 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:33.906505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c172c-db28-4c09-b79d-84be0d23d9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:33.907377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c172c-db28-4c09-b79d-84be0d23d9fb None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:33.907666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c172c-db28-4c09-b79d-84be0d23d9fb None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:33.907952 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c172c-db28-4c09-b79d-84be0d23d9fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:33.919617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c172c-db28-4c09-b79d-84be0d23d9fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-f88c172c-db28-4c09-b79d-84be0d23d9fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:33.928502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:33.929406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:33.930147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:33.930147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:33.930147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:33.930147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:33.930147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:33.930147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:33.930147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:33.930147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:33.930147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:33.930147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:33.930147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:37:33.930147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:37:33.930147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 24/49] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:37:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:33.935558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720ee02-e640-4fde-8db6-8e7283d948aa None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:34.029637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720ee02-e640-4fde-8db6-8e7283d948aa None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:34.061919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa303fe5-57f3-40f7-8f50-9cb901c908d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:34.062584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa303fe5-57f3-40f7-8f50-9cb901c908d0 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:34.062904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa303fe5-57f3-40f7-8f50-9cb901c908d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:34.063709 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa303fe5-57f3-40f7-8f50-9cb901c908d0 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:34.064458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa303fe5-57f3-40f7-8f50-9cb901c908d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:34.064765 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa303fe5-57f3-40f7-8f50-9cb901c908d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:34.066158 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa303fe5-57f3-40f7-8f50-9cb901c908d0 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:34.068781 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1011d-b2f7-45f2-b3af-39d9b024abdd None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:34.069075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1011d-b2f7-45f2-b3af-39d9b024abdd None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:34.069363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1011d-b2f7-45f2-b3af-39d9b024abdd None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:34.070769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 25/50] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:37:34.074486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 26/51] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:34.075945 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1866a-e573-42ad-98d9-67bb7ca55e91 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:34.076244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1866a-e573-42ad-98d9-67bb7ca55e91 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:34.076527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1866a-e573-42ad-98d9-67bb7ca55e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:34.080207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa303fe5-57f3-40f7-8f50-9cb901c908d0 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:34.093927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-93d1866a-e573-42ad-98d9-67bb7ca55e91 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:34.095496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-93d1866a-e573-42ad-98d9-67bb7ca55e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:34.160268 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 26/52] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:37:34.172722 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa303fe5-57f3-40f7-8f50-9cb901c908d0 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:34.182950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1866a-e573-42ad-98d9-67bb7ca55e91 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:34.204784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e45b70-7cb5-4e22-bc6d-bc15b858fd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:34.205646 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e45b70-7cb5-4e22-bc6d-bc15b858fd9d None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:34.205927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e45b70-7cb5-4e22-bc6d-bc15b858fd9d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:34.206202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e45b70-7cb5-4e22-bc6d-bc15b858fd9d None admin] PATH_INFO: `/v3/users/3b8d43206c7f4426b0be3e6d78690460` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:34.207418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e45b70-7cb5-4e22-bc6d-bc15b858fd9d None admin] RBAC: Authorizing `identity:get_user(user_id=3b8d43206c7f4426b0be3e6d78690460)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:34.208856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e45b70-7cb5-4e22-bc6d-bc15b858fd9d None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:34.210667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 27/53] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:34 2022] GET /identity/v3/users/3b8d43206c7f4426b0be3e6d78690460 => generated 247 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:34.215731 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e45b70-7cb5-4e22-bc6d-bc15b858fd9d None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:34.277638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1866a-e573-42ad-98d9-67bb7ca55e91 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:34.307775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e45b70-7cb5-4e22-bc6d-bc15b858fd9d None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:34.313814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228d925-8801-4bb4-aa47-ce1d2429720f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:34.314604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228d925-8801-4bb4-aa47-ce1d2429720f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:34.314924 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228d925-8801-4bb4-aa47-ce1d2429720f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:34.315203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228d925-8801-4bb4-aa47-ce1d2429720f None admin] PATH_INFO: `/v3/domains/default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:34.316383 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228d925-8801-4bb4-aa47-ce1d2429720f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:34.317843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228d925-8801-4bb4-aa47-ce1d2429720f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:34.319496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 27/54] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:37:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:34.324893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228d925-8801-4bb4-aa47-ce1d2429720f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:34.339491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7c860-f7dd-4357-83da-e303285f0bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:34.340277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7c860-f7dd-4357-83da-e303285f0bea None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:34.340563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7c860-f7dd-4357-83da-e303285f0bea None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:34.340868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7c860-f7dd-4357-83da-e303285f0bea None admin] PATH_INFO: `/v3/domains/default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:34.342039 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7c860-f7dd-4357-83da-e303285f0bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:34.343527 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7c860-f7dd-4357-83da-e303285f0bea None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:34.345172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 28/55] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:37:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:34.350605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7c860-f7dd-4357-83da-e303285f0bea None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:34.417321 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228d925-8801-4bb4-aa47-ce1d2429720f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:34.442945 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7c860-f7dd-4357-83da-e303285f0bea None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:34.449912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6ad5b-fc01-48f8-9e75-1368888698bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:34.450575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6ad5b-fc01-48f8-9e75-1368888698bd None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:34.450903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6ad5b-fc01-48f8-9e75-1368888698bd None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:34.451190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6ad5b-fc01-48f8-9e75-1368888698bd None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:34.451928 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6ad5b-fc01-48f8-9e75-1368888698bd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:34.453388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6ad5b-fc01-48f8-9e75-1368888698bd None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:34.474081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d58f0a-9132-40db-bfc2-d7616e6b137b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:34.474754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d58f0a-9132-40db-bfc2-d7616e6b137b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:34.475055 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d58f0a-9132-40db-bfc2-d7616e6b137b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:34.475347 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d58f0a-9132-40db-bfc2-d7616e6b137b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:34.476155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d58f0a-9132-40db-bfc2-d7616e6b137b None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.domain.id=default, user.id=3b8d43206c7f4426b0be3e6d78690460) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:34.476468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d58f0a-9132-40db-bfc2-d7616e6b137b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:34.477886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d58f0a-9132-40db-bfc2-d7616e6b137b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:34.479416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 28/56] 10.222.0.35 () {60 vars in 1140 bytes} [Fri May 13 21:37:34 2022] POST /identity/v3/projects => generated 298 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:37:34.483857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 29/57] 10.222.0.35 () {56 vars in 1322 bytes} [Fri May 13 21:37:34 2022] GET /identity/v3/role_assignments?user.id=3b8d43206c7f4426b0be3e6d78690460&scope.domain.id=default&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 530 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:36.703605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58219374-0585-4874-94f7-f23daeb3e009 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:36.704124 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58219374-0585-4874-94f7-f23daeb3e009 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:36.704204 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58219374-0585-4874-94f7-f23daeb3e009 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:36.705529 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 29/58] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:36.710766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6230690-2693-4550-ae54-5ad7e699acd3 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:36.711177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6230690-2693-4550-ae54-5ad7e699acd3 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:36.711333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6230690-2693-4550-ae54-5ad7e699acd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:36.727421 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-f6230690-2693-4550-ae54-5ad7e699acd3 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:36.728606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f6230690-2693-4550-ae54-5ad7e699acd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:36.791496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 30/59] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:36.816896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58219374-0585-4874-94f7-f23daeb3e009 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:36.912935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58219374-0585-4874-94f7-f23daeb3e009 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:36.950267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f60ce2-ed7a-4d3b-b5ba-ba0519c283af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:36.951087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f60ce2-ed7a-4d3b-b5ba-ba0519c283af None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:36.951372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f60ce2-ed7a-4d3b-b5ba-ba0519c283af None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:36.951649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f60ce2-ed7a-4d3b-b5ba-ba0519c283af None admin] PATH_INFO: `/v3/domains/default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:36.952817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f60ce2-ed7a-4d3b-b5ba-ba0519c283af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:36.954351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f60ce2-ed7a-4d3b-b5ba-ba0519c283af None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:36.955948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 30/60] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:37:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:36.961544 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6230690-2693-4550-ae54-5ad7e699acd3 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:37.053081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6230690-2693-4550-ae54-5ad7e699acd3 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:37.083631 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ce5eb-d67f-453d-b4ef-d1d4406d6f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:37.084281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ce5eb-d67f-453d-b4ef-d1d4406d6f31 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:37.084565 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ce5eb-d67f-453d-b4ef-d1d4406d6f31 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:37.084837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ce5eb-d67f-453d-b4ef-d1d4406d6f31 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:37.085538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ce5eb-d67f-453d-b4ef-d1d4406d6f31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:37.087017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ce5eb-d67f-453d-b4ef-d1d4406d6f31 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:37.112968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 31/61] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:37:36 2022] POST /identity/v3/users => generated 275 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:37:40.636635 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615d0d5-bc93-438e-857f-bbfe1ed42513 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:40.637695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615d0d5-bc93-438e-857f-bbfe1ed42513 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:40.638211 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615d0d5-bc93-438e-857f-bbfe1ed42513 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:40.639831 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 31/62] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:40.646138 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f937bae-7735-4e03-b522-19b63ec069df None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:40.647024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f937bae-7735-4e03-b522-19b63ec069df None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:40.647506 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f937bae-7735-4e03-b522-19b63ec069df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:40.671674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-9f937bae-7735-4e03-b522-19b63ec069df None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:40.673226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9f937bae-7735-4e03-b522-19b63ec069df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:40.687032 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d1f27-ba5b-4ec6-bf0f-0f047d160b28 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:40.687628 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d1f27-ba5b-4ec6-bf0f-0f047d160b28 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:40.688636 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d1f27-ba5b-4ec6-bf0f-0f047d160b28 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:40.690239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 32/63] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:40.695937 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c1d9f-e62f-4d43-97d2-3181d01ec34f None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:40.696223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c1d9f-e62f-4d43-97d2-3181d01ec34f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:40.696722 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c1d9f-e62f-4d43-97d2-3181d01ec34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:40.714093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-024c1d9f-e62f-4d43-97d2-3181d01ec34f None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:40.715644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-024c1d9f-e62f-4d43-97d2-3181d01ec34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:40.749635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 32/64] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:40.751549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b52023-fdfd-4ea8-af5f-65942d551a8f None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:40.752029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b52023-fdfd-4ea8-af5f-65942d551a8f None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:40.752439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b52023-fdfd-4ea8-af5f-65942d551a8f None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:40.754119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 33/65] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:40.759656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c96186-ca83-476e-a818-15517af9088e None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:40.760110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c96186-ca83-476e-a818-15517af9088e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:40.760552 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c96186-ca83-476e-a818-15517af9088e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:40.777654 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-20c96186-ca83-476e-a818-15517af9088e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:40.779091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-20c96186-ca83-476e-a818-15517af9088e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:40.785239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 33/66] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:37:40.792172 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c1d9f-e62f-4d43-97d2-3181d01ec34f None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:40.850899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 34/67] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:37:40.851760 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c96186-ca83-476e-a818-15517af9088e None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:40.913629 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c1d9f-e62f-4d43-97d2-3181d01ec34f None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:40.949311 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9e6a5-0331-4fd3-b0ea-1615cb522250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:40.950486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9e6a5-0331-4fd3-b0ea-1615cb522250 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:40.951000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9e6a5-0331-4fd3-b0ea-1615cb522250 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:40.951475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9e6a5-0331-4fd3-b0ea-1615cb522250 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:40.957787 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c96186-ca83-476e-a818-15517af9088e None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:40.963919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9e6a5-0331-4fd3-b0ea-1615cb522250 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-5ae9e6a5-0331-4fd3-b0ea-1615cb522250 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:40.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:40.973710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:40.974475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:40.974475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:40.974475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:40.974475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:40.974475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:40.974475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:40.974475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:40.974475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:40.974475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:40.974475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:40.974475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:37:40.974475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:37:40.977036 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 34/68] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:37:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:37:40.978123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9e6a5-0331-4fd3-b0ea-1615cb522250 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:40.999626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcfa97-e505-49f6-b752-4fdd2ba3ed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:41.001297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcfa97-e505-49f6-b752-4fdd2ba3ed46 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.001734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcfa97-e505-49f6-b752-4fdd2ba3ed46 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.002173 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcfa97-e505-49f6-b752-4fdd2ba3ed46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.013842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcfa97-e505-49f6-b752-4fdd2ba3ed46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-b0fcfa97-e505-49f6-b752-4fdd2ba3ed46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:41.021653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:41.022489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:41.023302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:41.023302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:41.023302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:41.023302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:41.023302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:41.023302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:41.023302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:41.023302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:41.023302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:41.023302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:41.023302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:37:41.023302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:37:41.023302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 35/69] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:41.027538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcfa97-e505-49f6-b752-4fdd2ba3ed46 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:41.082989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9e6a5-0331-4fd3-b0ea-1615cb522250 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:41.115344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34fea1-31c7-476c-9d80-d06614c84058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:41.116389 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34fea1-31c7-476c-9d80-d06614c84058 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.116829 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34fea1-31c7-476c-9d80-d06614c84058 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.117107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34fea1-31c7-476c-9d80-d06614c84058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.127203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcfa97-e505-49f6-b752-4fdd2ba3ed46 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:41.127958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34fea1-31c7-476c-9d80-d06614c84058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-fe34fea1-31c7-476c-9d80-d06614c84058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:41.135771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:41.136606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:41.137353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:41.137353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:41.137353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:41.137353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:41.137353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:41.137353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:41.137353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:41.137353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:41.137353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:41.137353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:41.137353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:37:41.137353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:37:41.137353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 35/70] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:37:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:41.138588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93b38f-293e-46a7-b76c-f6bac827b1a7 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.138588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93b38f-293e-46a7-b76c-f6bac827b1a7 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.138784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93b38f-293e-46a7-b76c-f6bac827b1a7 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.140207 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 36/71] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:41.142107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93b38f-293e-46a7-b76c-f6bac827b1a7 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:41.170530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b1cbd-dcef-4ae7-8ac7-280956cc9ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:41.171175 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b1cbd-dcef-4ae7-8ac7-280956cc9ce8 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.171466 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b1cbd-dcef-4ae7-8ac7-280956cc9ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.171749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b1cbd-dcef-4ae7-8ac7-280956cc9ce8 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.173039 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b1cbd-dcef-4ae7-8ac7-280956cc9ce8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:41.173351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b1cbd-dcef-4ae7-8ac7-280956cc9ce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:41.174773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b1cbd-dcef-4ae7-8ac7-280956cc9ce8 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:41.182724 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 36/72] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:37:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:41.183405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b1cbd-dcef-4ae7-8ac7-280956cc9ce8 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:41.242289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93b38f-293e-46a7-b76c-f6bac827b1a7 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:41.274712 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4bca4-78c9-47d4-8daa-6766191e7710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:41.275394 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4bca4-78c9-47d4-8daa-6766191e7710 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.275684 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4bca4-78c9-47d4-8daa-6766191e7710 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.275968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4bca4-78c9-47d4-8daa-6766191e7710 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.276726 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4bca4-78c9-47d4-8daa-6766191e7710 None admin] RBAC: Adding query filter params (name=member) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:41.277043 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4bca4-78c9-47d4-8daa-6766191e7710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:41.278447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4bca4-78c9-47d4-8daa-6766191e7710 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:41.279954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b1cbd-dcef-4ae7-8ac7-280956cc9ce8 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:41.287695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 37/73] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:37:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:41.292323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae48593-b298-4c08-9a17-56755dfbebcd None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.292323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae48593-b298-4c08-9a17-56755dfbebcd None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.292323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae48593-b298-4c08-9a17-56755dfbebcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.305794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5ae48593-b298-4c08-9a17-56755dfbebcd None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:41.307035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5ae48593-b298-4c08-9a17-56755dfbebcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:41.317475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f256e-b445-4d89-8b4e-a50347fc3170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:41.318140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f256e-b445-4d89-8b4e-a50347fc3170 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.318427 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f256e-b445-4d89-8b4e-a50347fc3170 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.318730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f256e-b445-4d89-8b4e-a50347fc3170 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.319475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f256e-b445-4d89-8b4e-a50347fc3170 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:41.319784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f256e-b445-4d89-8b4e-a50347fc3170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:41.321191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f256e-b445-4d89-8b4e-a50347fc3170 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:41.329059 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 37/74] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:41.330247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d79b3-16c3-4293-bdea-9cf4f28f9534 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.330540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d79b3-16c3-4293-bdea-9cf4f28f9534 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.330852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d79b3-16c3-4293-bdea-9cf4f28f9534 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.332091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 38/75] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:41.332786 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d79b3-16c3-4293-bdea-9cf4f28f9534 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:41.373030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 38/76] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:41.373774 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae48593-b298-4c08-9a17-56755dfbebcd None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:41.431346 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d79b3-16c3-4293-bdea-9cf4f28f9534 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:41.462394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97626715-a05f-4d4e-a5c9-57a5c77cc59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:41.463296 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97626715-a05f-4d4e-a5c9-57a5c77cc59d None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.463586 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97626715-a05f-4d4e-a5c9-57a5c77cc59d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.463868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97626715-a05f-4d4e-a5c9-57a5c77cc59d None admin] PATH_INFO: `/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.467674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae48593-b298-4c08-9a17-56755dfbebcd None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:41.476954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97626715-a05f-4d4e-a5c9-57a5c77cc59d None admin] RBAC: Authorizing `identity:get_user(user_id=c5810ecbb17a4b1ea3085dd850c6fa2d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:41.478427 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97626715-a05f-4d4e-a5c9-57a5c77cc59d None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:41.480112 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 39/77] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:41 2022] GET /identity/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:41.482042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97626715-a05f-4d4e-a5c9-57a5c77cc59d None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:41.501080 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ad8e3-0613-4a70-ad73-101c5d1131f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:41.501947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ad8e3-0613-4a70-ad73-101c5d1131f1 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.502238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ad8e3-0613-4a70-ad73-101c5d1131f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.502518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ad8e3-0613-4a70-ad73-101c5d1131f1 None admin] PATH_INFO: `/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.504019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ad8e3-0613-4a70-ad73-101c5d1131f1 None admin] RBAC: Authorizing `identity:get_user(user_id=c5810ecbb17a4b1ea3085dd850c6fa2d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:41.505485 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ad8e3-0613-4a70-ad73-101c5d1131f1 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:41.507157 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 39/78] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:41 2022] GET /identity/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d => generated 275 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:41.508283 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969f8a8-d3df-44ec-b952-653f4f5f3eab None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.508571 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969f8a8-d3df-44ec-b952-653f4f5f3eab None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.508854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969f8a8-d3df-44ec-b952-653f4f5f3eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.524223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-b969f8a8-d3df-44ec-b952-653f4f5f3eab None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:41.525415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b969f8a8-d3df-44ec-b952-653f4f5f3eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:41.579573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97626715-a05f-4d4e-a5c9-57a5c77cc59d None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:41.587550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 40/79] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:41.596378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969f8a8-d3df-44ec-b952-653f4f5f3eab None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:41.621327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9968bea-d2e7-4d2d-867f-6fb35a3a33ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:41.622199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9968bea-d2e7-4d2d-867f-6fb35a3a33ce None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.622483 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9968bea-d2e7-4d2d-867f-6fb35a3a33ce None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.622838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9968bea-d2e7-4d2d-867f-6fb35a3a33ce None admin] PATH_INFO: `/v3/users/3b8d43206c7f4426b0be3e6d78690460` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.624025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9968bea-d2e7-4d2d-867f-6fb35a3a33ce None admin] RBAC: Authorizing `identity:get_user(user_id=3b8d43206c7f4426b0be3e6d78690460)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:41.625475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9968bea-d2e7-4d2d-867f-6fb35a3a33ce None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:41.627134 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 40/80] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:41 2022] GET /identity/v3/users/3b8d43206c7f4426b0be3e6d78690460 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:41.627894 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9968bea-d2e7-4d2d-867f-6fb35a3a33ce None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:41.685894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969f8a8-d3df-44ec-b952-653f4f5f3eab None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:41.717573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd997682-65dc-4b53-8573-c7f8e01d563f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:41.718353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd997682-65dc-4b53-8573-c7f8e01d563f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.718639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd997682-65dc-4b53-8573-c7f8e01d563f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.718944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd997682-65dc-4b53-8573-c7f8e01d563f None admin] PATH_INFO: `/v3/roles/member` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.721115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9968bea-d2e7-4d2d-867f-6fb35a3a33ce None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:41.729178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd997682-65dc-4b53-8573-c7f8e01d563f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-fd997682-65dc-4b53-8573-c7f8e01d563f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:41.736782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:41.737639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:41.743162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:41.743162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:41.743162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:41.743162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:41.743162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:41.743162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:41.743162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:41.743162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:41.743162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:41.743162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:41.743162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:37:41.743162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:37:41.743162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 41/81] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:41.743162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd997682-65dc-4b53-8573-c7f8e01d563f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:41.752722 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe7575-8a09-432a-b6a1-1484a34fea2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:41.753616 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe7575-8a09-432a-b6a1-1484a34fea2b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.753901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe7575-8a09-432a-b6a1-1484a34fea2b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.754178 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe7575-8a09-432a-b6a1-1484a34fea2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.817029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe7575-8a09-432a-b6a1-1484a34fea2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:41.818497 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe7575-8a09-432a-b6a1-1484a34fea2b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:41.832365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd997682-65dc-4b53-8573-c7f8e01d563f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:41.865239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61821f-cf24-4b07-9aec-9c8eddf20525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:41.865879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61821f-cf24-4b07-9aec-9c8eddf20525 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:41.866172 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61821f-cf24-4b07-9aec-9c8eddf20525 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:41.866453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61821f-cf24-4b07-9aec-9c8eddf20525 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:41.910218 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 41/82] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:37:41.911069 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe7575-8a09-432a-b6a1-1484a34fea2b None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:41.930007 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61821f-cf24-4b07-9aec-9c8eddf20525 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:41.931534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61821f-cf24-4b07-9aec-9c8eddf20525 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:42.001389 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe7575-8a09-432a-b6a1-1484a34fea2b None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:42.027741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 42/83] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:37:42.027741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61821f-cf24-4b07-9aec-9c8eddf20525 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:42.032527 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fdf1b-b95f-4c07-8eeb-24a3c4fe0c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:42.033308 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fdf1b-b95f-4c07-8eeb-24a3c4fe0c12 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:42.033590 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fdf1b-b95f-4c07-8eeb-24a3c4fe0c12 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:42.033865 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fdf1b-b95f-4c07-8eeb-24a3c4fe0c12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:42.035037 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fdf1b-b95f-4c07-8eeb-24a3c4fe0c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:42.036690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fdf1b-b95f-4c07-8eeb-24a3c4fe0c12 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:42.038071 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 42/84] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:37:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:42.038835 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fdf1b-b95f-4c07-8eeb-24a3c4fe0c12 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:42.115288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61821f-cf24-4b07-9aec-9c8eddf20525 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:42.131087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fdf1b-b95f-4c07-8eeb-24a3c4fe0c12 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:42.146800 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6f966-d587-4eb8-96d2-a66ee7e30839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:42.147442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6f966-d587-4eb8-96d2-a66ee7e30839 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:42.147732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6f966-d587-4eb8-96d2-a66ee7e30839 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:42.148010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6f966-d587-4eb8-96d2-a66ee7e30839 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:42.164083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444111b-e64c-445a-9df5-36a78d2de9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:42.164743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444111b-e64c-445a-9df5-36a78d2de9c8 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:42.165027 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444111b-e64c-445a-9df5-36a78d2de9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:42.165305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444111b-e64c-445a-9df5-36a78d2de9c8 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:42.166044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444111b-e64c-445a-9df5-36a78d2de9c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:42.166347 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444111b-e64c-445a-9df5-36a78d2de9c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:42.167776 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444111b-e64c-445a-9df5-36a78d2de9c8 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:42.176142 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 43/85] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:37:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:42.176911 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444111b-e64c-445a-9df5-36a78d2de9c8 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:42.212730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6f966-d587-4eb8-96d2-a66ee7e30839 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:42.214196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6f966-d587-4eb8-96d2-a66ee7e30839 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:42.268806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444111b-e64c-445a-9df5-36a78d2de9c8 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:42.307160 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8a5ba-8f7d-42ab-8822-c528413a6b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:42.308287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8a5ba-8f7d-42ab-8822-c528413a6b5f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:42.308687 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8a5ba-8f7d-42ab-8822-c528413a6b5f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:42.309054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8a5ba-8f7d-42ab-8822-c528413a6b5f None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:42.310356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8a5ba-8f7d-42ab-8822-c528413a6b5f None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:42.310588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 43/86] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:37:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:37:42.311480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6f966-d587-4eb8-96d2-a66ee7e30839 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:42.317998 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8a5ba-8f7d-42ab-8822-c528413a6b5f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:42.325351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 44/87] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:42 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:42.325351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8a5ba-8f7d-42ab-8822-c528413a6b5f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:42.404297 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6f966-d587-4eb8-96d2-a66ee7e30839 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:42.413149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8a5ba-8f7d-42ab-8822-c528413a6b5f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:42.436460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf81b9-8bc4-4459-b10a-7b6a933e33ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:42.437300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf81b9-8bc4-4459-b10a-7b6a933e33ff None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:42.437584 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf81b9-8bc4-4459-b10a-7b6a933e33ff None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:42.437889 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf81b9-8bc4-4459-b10a-7b6a933e33ff None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:42.439106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf81b9-8bc4-4459-b10a-7b6a933e33ff None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:42.440562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf81b9-8bc4-4459-b10a-7b6a933e33ff None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:42.442287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 44/88] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:42 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:42.443073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf81b9-8bc4-4459-b10a-7b6a933e33ff None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:42.448259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7eaf1-0e30-4e92-8c1d-9ab16da16114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:42.448916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7eaf1-0e30-4e92-8c1d-9ab16da16114 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:42.449200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7eaf1-0e30-4e92-8c1d-9ab16da16114 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:42.449476 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7eaf1-0e30-4e92-8c1d-9ab16da16114 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:42.450175 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7eaf1-0e30-4e92-8c1d-9ab16da16114 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:42.451642 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7eaf1-0e30-4e92-8c1d-9ab16da16114 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:42.472096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 45/89] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:37:42 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:37:42.472897 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7eaf1-0e30-4e92-8c1d-9ab16da16114 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:42.553718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf81b9-8bc4-4459-b10a-7b6a933e33ff None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:42.579021 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7eaf1-0e30-4e92-8c1d-9ab16da16114 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:42.590620 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77be57a-2f8d-44f3-bae9-4d0bbcd2b878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:42.591431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77be57a-2f8d-44f3-bae9-4d0bbcd2b878 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:42.591717 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77be57a-2f8d-44f3-bae9-4d0bbcd2b878 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:42.592045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77be57a-2f8d-44f3-bae9-4d0bbcd2b878 None admin] PATH_INFO: `/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:42.593299 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77be57a-2f8d-44f3-bae9-4d0bbcd2b878 None admin] RBAC: Authorizing `identity:get_user(user_id=c5810ecbb17a4b1ea3085dd850c6fa2d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:42.595372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77be57a-2f8d-44f3-bae9-4d0bbcd2b878 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:42.601604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 45/90] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:42 2022] GET /identity/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:42.601604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77be57a-2f8d-44f3-bae9-4d0bbcd2b878 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:42.618378 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da3baa-dffb-453d-af1f-a4bfe928983c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:42.619295 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da3baa-dffb-453d-af1f-a4bfe928983c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:42.619580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da3baa-dffb-453d-af1f-a4bfe928983c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:42.619875 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da3baa-dffb-453d-af1f-a4bfe928983c None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:42.621162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da3baa-dffb-453d-af1f-a4bfe928983c None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:42.622652 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da3baa-dffb-453d-af1f-a4bfe928983c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:42.624458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 46/91] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:42 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:42.625251 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da3baa-dffb-453d-af1f-a4bfe928983c None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:42.704850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77be57a-2f8d-44f3-bae9-4d0bbcd2b878 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:42.725568 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da3baa-dffb-453d-af1f-a4bfe928983c None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:42.742457 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326fae3-97c9-428b-ba6f-1cf2bf7b2523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:42.743480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326fae3-97c9-428b-ba6f-1cf2bf7b2523 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:42.744031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326fae3-97c9-428b-ba6f-1cf2bf7b2523 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:42.744486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326fae3-97c9-428b-ba6f-1cf2bf7b2523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:42.745342 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326fae3-97c9-428b-ba6f-1cf2bf7b2523 None admin] RBAC: Adding query filter params (role.id=a29f4364896c46dd84dcee5b969b9898, scope.project.id=80047f4aaba64866971a8b97d21dee0a, user.id=c5810ecbb17a4b1ea3085dd850c6fa2d) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:42.745663 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326fae3-97c9-428b-ba6f-1cf2bf7b2523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:42.747231 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326fae3-97c9-428b-ba6f-1cf2bf7b2523 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:42.756322 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 46/92] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:37:42 2022] GET /identity/v3/role_assignments?user.id=c5810ecbb17a4b1ea3085dd850c6fa2d&scope.project.id=80047f4aaba64866971a8b97d21dee0a&role.id=a29f4364896c46dd84dcee5b969b9898 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:42.757158 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326fae3-97c9-428b-ba6f-1cf2bf7b2523 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:42.773372 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921ac1c-52ed-43f9-af84-c542c9e383ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:42.774094 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921ac1c-52ed-43f9-af84-c542c9e383ce None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:42.774394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921ac1c-52ed-43f9-af84-c542c9e383ce None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:42.774677 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921ac1c-52ed-43f9-af84-c542c9e383ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:42.775680 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921ac1c-52ed-43f9-af84-c542c9e383ce None admin] RBAC: Adding query filter params (role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d, scope.project.id=80047f4aaba64866971a8b97d21dee0a, user.id=c5810ecbb17a4b1ea3085dd850c6fa2d) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:42.776001 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921ac1c-52ed-43f9-af84-c542c9e383ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:42.777697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921ac1c-52ed-43f9-af84-c542c9e383ce None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:42.786445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 47/93] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:37:42 2022] GET /identity/v3/role_assignments?user.id=c5810ecbb17a4b1ea3085dd850c6fa2d&scope.project.id=80047f4aaba64866971a8b97d21dee0a&role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:42.787398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921ac1c-52ed-43f9-af84-c542c9e383ce None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:42.920370 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326fae3-97c9-428b-ba6f-1cf2bf7b2523 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:42.940227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921ac1c-52ed-43f9-af84-c542c9e383ce None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:42.969335 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bf8ec-7986-483c-94dc-5f8e2f1bdf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:42.970631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bf8ec-7986-483c-94dc-5f8e2f1bdf71 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:42.971531 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bf8ec-7986-483c-94dc-5f8e2f1bdf71 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:42.972277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bf8ec-7986-483c-94dc-5f8e2f1bdf71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:42.982370 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6533ee8-a3d0-4ec7-b6f1-6f01be3d62f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:42.983167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6533ee8-a3d0-4ec7-b6f1-6f01be3d62f3 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:42.983526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6533ee8-a3d0-4ec7-b6f1-6f01be3d62f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:42.983881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6533ee8-a3d0-4ec7-b6f1-6f01be3d62f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:42.985010 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6533ee8-a3d0-4ec7-b6f1-6f01be3d62f3 None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=80047f4aaba64866971a8b97d21dee0a, user.id=3b8d43206c7f4426b0be3e6d78690460) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:42.985177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6533ee8-a3d0-4ec7-b6f1-6f01be3d62f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:42.986874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6533ee8-a3d0-4ec7-b6f1-6f01be3d62f3 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:42.993202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 48/94] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:37:42 2022] GET /identity/v3/role_assignments?user.id=3b8d43206c7f4426b0be3e6d78690460&scope.project.id=80047f4aaba64866971a8b97d21dee0a&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:43.056351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bf8ec-7986-483c-94dc-5f8e2f1bdf71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:43.058291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bf8ec-7986-483c-94dc-5f8e2f1bdf71 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:43.170360 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 47/95] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:37:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:37:43.176222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6533ee8-a3d0-4ec7-b6f1-6f01be3d62f3 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:43.305129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6533ee8-a3d0-4ec7-b6f1-6f01be3d62f3 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:43.345815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4296a-c830-415b-8813-a3a4cbc8aa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:43.346689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4296a-c830-415b-8813-a3a4cbc8aa3d None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:43.347010 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4296a-c830-415b-8813-a3a4cbc8aa3d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:43.347291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4296a-c830-415b-8813-a3a4cbc8aa3d None admin] PATH_INFO: `/v3/projects/7bf51f2a7fbd437e8dd20b79f52ea901` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:43.351418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4296a-c830-415b-8813-a3a4cbc8aa3d None admin] RBAC: Authorizing `identity:get_project(project_id=7bf51f2a7fbd437e8dd20b79f52ea901)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:43.353029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4296a-c830-415b-8813-a3a4cbc8aa3d None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:43.354879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 49/96] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:43 2022] GET /identity/v3/projects/7bf51f2a7fbd437e8dd20b79f52ea901 => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:43.361288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bf8ec-7986-483c-94dc-5f8e2f1bdf71 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:43.462653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bf8ec-7986-483c-94dc-5f8e2f1bdf71 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:43.502769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff184d-c09c-4be4-a3e3-2cea5671d3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:43.503695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff184d-c09c-4be4-a3e3-2cea5671d3f7 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:43.503981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff184d-c09c-4be4-a3e3-2cea5671d3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:43.504263 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff184d-c09c-4be4-a3e3-2cea5671d3f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:43.505104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff184d-c09c-4be4-a3e3-2cea5671d3f7 None admin] RBAC: Adding query filter params (role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d, scope.project.id=7bf51f2a7fbd437e8dd20b79f52ea901, user.id=c5810ecbb17a4b1ea3085dd850c6fa2d) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:43.505417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff184d-c09c-4be4-a3e3-2cea5671d3f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:43.506911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff184d-c09c-4be4-a3e3-2cea5671d3f7 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:43.512864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 48/97] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:37:43 2022] GET /identity/v3/role_assignments?user.id=c5810ecbb17a4b1ea3085dd850c6fa2d&scope.project.id=7bf51f2a7fbd437e8dd20b79f52ea901&role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:45.740878 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc613647-164f-4149-beef-5cb634227fb6 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:45.741880 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc613647-164f-4149-beef-5cb634227fb6 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:45.742333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc613647-164f-4149-beef-5cb634227fb6 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:45.744046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 50/98] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:45.750195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcbaff-fa38-4aa2-9316-ab2fe0ca0175 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:45.751144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcbaff-fa38-4aa2-9316-ab2fe0ca0175 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:45.751586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcbaff-fa38-4aa2-9316-ab2fe0ca0175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:45.776675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-cddcbaff-fa38-4aa2-9316-ab2fe0ca0175 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:45.778212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-cddcbaff-fa38-4aa2-9316-ab2fe0ca0175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:45.861250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 49/99] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:45.895928 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc613647-164f-4149-beef-5cb634227fb6 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:45.945307 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a24b14-69b4-40f8-a39b-05e18bc335a2 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:45.945634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a24b14-69b4-40f8-a39b-05e18bc335a2 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:45.945872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a24b14-69b4-40f8-a39b-05e18bc335a2 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:45.947294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 50/100] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:37:45.960793 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6b005-7bcf-465a-b328-fe08de1a4299 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:45.961076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6b005-7bcf-465a-b328-fe08de1a4299 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:45.961355 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6b005-7bcf-465a-b328-fe08de1a4299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:45.987049 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-f9a6b005-7bcf-465a-b328-fe08de1a4299 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:45.988256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f9a6b005-7bcf-465a-b328-fe08de1a4299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:46.087654 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc613647-164f-4149-beef-5cb634227fb6 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:46.125862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 51/101] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:37:46.127053 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd2994-742f-46ad-8508-3f267c9f3ea9 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:46.127349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd2994-742f-46ad-8508-3f267c9f3ea9 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:46.127631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd2994-742f-46ad-8508-3f267c9f3ea9 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:46.133574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 52/102] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:46.134686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff66a3-2c67-4c4f-85f4-ac3f9b1b9b06 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:46.134999 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff66a3-2c67-4c4f-85f4-ac3f9b1b9b06 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:46.135284 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff66a3-2c67-4c4f-85f4-ac3f9b1b9b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:46.158725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dff3b-f81c-4f94-ac49-d0bf894bb0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:46.158725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dff3b-f81c-4f94-ac49-d0bf894bb0b5 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:46.158725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dff3b-f81c-4f94-ac49-d0bf894bb0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:46.158725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dff3b-f81c-4f94-ac49-d0bf894bb0b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:46.170953 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-eaff66a3-2c67-4c4f-85f4-ac3f9b1b9b06 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:46.172225 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-eaff66a3-2c67-4c4f-85f4-ac3f9b1b9b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:46.180532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997dff3b-f81c-4f94-ac49-d0bf894bb0b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-997dff3b-f81c-4f94-ac49-d0bf894bb0b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:46.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:46.201767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:46.202635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:46.202635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:46.202635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:46.202635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:46.202635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:46.202635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:46.202635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:46.202635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:46.202635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:46.202635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:46.202635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:37:46.202635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:37:46.202635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 51/103] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:46.203937 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dff3b-f81c-4f94-ac49-d0bf894bb0b5 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:46.271114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 53/104] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:46.272272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff66a3-2c67-4c4f-85f4-ac3f9b1b9b06 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:46.330594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dff3b-f81c-4f94-ac49-d0bf894bb0b5 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:46.366336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7b483-5d77-4fcd-aea8-3b4d4ef547b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:46.367088 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7b483-5d77-4fcd-aea8-3b4d4ef547b4 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:46.367373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7b483-5d77-4fcd-aea8-3b4d4ef547b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:46.367652 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7b483-5d77-4fcd-aea8-3b4d4ef547b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:46.374471 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff66a3-2c67-4c4f-85f4-ac3f9b1b9b06 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:46.378882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7b483-5d77-4fcd-aea8-3b4d4ef547b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-2bd7b483-5d77-4fcd-aea8-3b4d4ef547b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:46.386580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:46.387703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:46.388657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:46.388657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:46.388657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:46.388657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:46.388657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:46.388657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:46.388657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:46.388657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:46.388657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:46.388657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:46.388657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:37:46.388657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:37:46.388657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 52/105] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:46.389321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2fee7-7936-44ac-8e81-221621969f5b None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:46.389321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2fee7-7936-44ac-8e81-221621969f5b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:46.394358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2fee7-7936-44ac-8e81-221621969f5b None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:46.394358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 53/106] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:46.394358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2fee7-7936-44ac-8e81-221621969f5b None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:46.420587 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cf134-47ad-4e84-a098-7cd90139ed04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:46.421313 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cf134-47ad-4e84-a098-7cd90139ed04 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:46.421604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cf134-47ad-4e84-a098-7cd90139ed04 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:46.421883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cf134-47ad-4e84-a098-7cd90139ed04 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:46.423231 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cf134-47ad-4e84-a098-7cd90139ed04 None admin] RBAC: Adding query filter params (name=member) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:46.423545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cf134-47ad-4e84-a098-7cd90139ed04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:46.424954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cf134-47ad-4e84-a098-7cd90139ed04 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:46.435133 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 54/107] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:37:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:46.435133 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c106fa-d23f-4f13-b6d6-ce2f5c16c64f None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:46.435133 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c106fa-d23f-4f13-b6d6-ce2f5c16c64f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:46.435133 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c106fa-d23f-4f13-b6d6-ce2f5c16c64f None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:46.436165 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 55/108] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:46.436855 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c106fa-d23f-4f13-b6d6-ce2f5c16c64f None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:46.495679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2fee7-7936-44ac-8e81-221621969f5b None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:46.530445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea6ac0-5687-4ee6-acda-fabb9660c482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:46.531279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea6ac0-5687-4ee6-acda-fabb9660c482 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:46.531596 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea6ac0-5687-4ee6-acda-fabb9660c482 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:46.531877 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea6ac0-5687-4ee6-acda-fabb9660c482 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:46.541877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c106fa-d23f-4f13-b6d6-ce2f5c16c64f None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:46.542103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea6ac0-5687-4ee6-acda-fabb9660c482 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-d4ea6ac0-5687-4ee6-acda-fabb9660c482 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:46.550857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:46.551704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:46.552455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:46.552455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:46.552455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:46.552455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:46.552455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:46.552455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:46.552455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:46.552455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:46.552455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:46.552455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:46.552455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:37:46.552455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:37:46.552455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 54/109] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:37:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:46.553630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d09463-6fee-4aa0-9739-105fd2352271 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:46.553630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d09463-6fee-4aa0-9739-105fd2352271 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:46.553876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d09463-6fee-4aa0-9739-105fd2352271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:46.571348 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-41d09463-6fee-4aa0-9739-105fd2352271 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:46.572528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-41d09463-6fee-4aa0-9739-105fd2352271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:46.579711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e897b48-f0c2-4a20-9e22-5072525ef817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:46.580392 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e897b48-f0c2-4a20-9e22-5072525ef817 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:46.580679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e897b48-f0c2-4a20-9e22-5072525ef817 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:46.580959 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e897b48-f0c2-4a20-9e22-5072525ef817 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:46.581710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e897b48-f0c2-4a20-9e22-5072525ef817 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:46.582029 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e897b48-f0c2-4a20-9e22-5072525ef817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:46.583446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e897b48-f0c2-4a20-9e22-5072525ef817 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:46.597099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 56/110] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:37:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:46.597099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e897b48-f0c2-4a20-9e22-5072525ef817 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:46.643289 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 55/111] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:46.644790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbf151-cf78-4625-9fdf-045377e52f89 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:46.645222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbf151-cf78-4625-9fdf-045377e52f89 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:46.645640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbf151-cf78-4625-9fdf-045377e52f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:46.661908 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-defbf151-cf78-4625-9fdf-045377e52f89 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:46.663396 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-defbf151-cf78-4625-9fdf-045377e52f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:46.711437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e897b48-f0c2-4a20-9e22-5072525ef817 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:46.728675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 56/112] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:46.739740 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbf151-cf78-4625-9fdf-045377e52f89 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:46.750969 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b5052-0429-44ed-90e1-280a3acb6bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:46.751847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b5052-0429-44ed-90e1-280a3acb6bcc None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:46.752138 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b5052-0429-44ed-90e1-280a3acb6bcc None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:46.752418 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b5052-0429-44ed-90e1-280a3acb6bcc None admin] PATH_INFO: `/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:46.753963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b5052-0429-44ed-90e1-280a3acb6bcc None admin] RBAC: Authorizing `identity:get_user(user_id=c5810ecbb17a4b1ea3085dd850c6fa2d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:46.755489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b5052-0429-44ed-90e1-280a3acb6bcc None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:46.757335 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 57/113] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:46 2022] GET /identity/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:46.757946 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b5052-0429-44ed-90e1-280a3acb6bcc None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:46.835890 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbf151-cf78-4625-9fdf-045377e52f89 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:46.854913 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b5052-0429-44ed-90e1-280a3acb6bcc None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:46.869764 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794e00d-5ede-404d-a819-6fcc108d79ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:46.870458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794e00d-5ede-404d-a819-6fcc108d79ae None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:46.870769 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794e00d-5ede-404d-a819-6fcc108d79ae None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:46.871062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794e00d-5ede-404d-a819-6fcc108d79ae None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:46.871828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794e00d-5ede-404d-a819-6fcc108d79ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:46.872140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794e00d-5ede-404d-a819-6fcc108d79ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:46.873606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794e00d-5ede-404d-a819-6fcc108d79ae None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:46.882892 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 57/114] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:37:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:46.884006 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794e00d-5ede-404d-a819-6fcc108d79ae None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:46.898471 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bc5a4-98f2-499f-8eb0-14133baedb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:46.899640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bc5a4-98f2-499f-8eb0-14133baedb4c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:46.899933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bc5a4-98f2-499f-8eb0-14133baedb4c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:46.900214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bc5a4-98f2-499f-8eb0-14133baedb4c None admin] PATH_INFO: `/v3/users/1984a1910cb74be89a85e92df2957b48` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:46.916414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bc5a4-98f2-499f-8eb0-14133baedb4c None admin] RBAC: Authorizing `identity:get_user(user_id=1984a1910cb74be89a85e92df2957b48)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:46.917881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bc5a4-98f2-499f-8eb0-14133baedb4c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:46.919989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 58/115] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:46 2022] GET /identity/v3/users/1984a1910cb74be89a85e92df2957b48 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:46.920777 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bc5a4-98f2-499f-8eb0-14133baedb4c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:46.984397 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794e00d-5ede-404d-a819-6fcc108d79ae None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:47.016983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88469dbd-2a18-4e39-93ad-a4f0244a48f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:47.017780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88469dbd-2a18-4e39-93ad-a4f0244a48f1 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:47.018068 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88469dbd-2a18-4e39-93ad-a4f0244a48f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:47.018353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88469dbd-2a18-4e39-93ad-a4f0244a48f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:47.021254 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bc5a4-98f2-499f-8eb0-14133baedb4c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:47.028600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88469dbd-2a18-4e39-93ad-a4f0244a48f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-88469dbd-2a18-4e39-93ad-a4f0244a48f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:47.036795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:47.037630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:47.043604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:47.043604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:47.043604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:47.043604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:47.043604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:47.043604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:47.043604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:47.043604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:47.043604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:47.043604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:47.043604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:37:47.043604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:37:47.043604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 58/116] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:37:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:47.043604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88469dbd-2a18-4e39-93ad-a4f0244a48f1 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:47.061580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8481016-9f5a-48b0-aefa-c928c53465f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:47.062436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8481016-9f5a-48b0-aefa-c928c53465f8 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:47.062779 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8481016-9f5a-48b0-aefa-c928c53465f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:47.063117 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8481016-9f5a-48b0-aefa-c928c53465f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:47.064428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8481016-9f5a-48b0-aefa-c928c53465f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:47.065944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8481016-9f5a-48b0-aefa-c928c53465f8 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:47.067774 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 59/117] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:37:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:47.068399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8481016-9f5a-48b0-aefa-c928c53465f8 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:47.134851 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88469dbd-2a18-4e39-93ad-a4f0244a48f1 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:47.162454 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8481016-9f5a-48b0-aefa-c928c53465f8 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:47.167923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3e73b-769e-46d1-aece-b024a6a9a113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:47.168777 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3e73b-769e-46d1-aece-b024a6a9a113 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:47.169066 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3e73b-769e-46d1-aece-b024a6a9a113 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:47.169346 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3e73b-769e-46d1-aece-b024a6a9a113 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:47.170519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3e73b-769e-46d1-aece-b024a6a9a113 None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:47.172024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3e73b-769e-46d1-aece-b024a6a9a113 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:47.173750 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 59/118] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:47 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:47.175456 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3e73b-769e-46d1-aece-b024a6a9a113 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:47.194416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068e6e7-fe08-49ec-8866-9cb5c76080df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:47.195311 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e6e7-fe08-49ec-8866-9cb5c76080df None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:47.195596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e6e7-fe08-49ec-8866-9cb5c76080df None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:47.195872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e6e7-fe08-49ec-8866-9cb5c76080df None admin] PATH_INFO: `/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:47.197051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068e6e7-fe08-49ec-8866-9cb5c76080df None admin] RBAC: Authorizing `identity:get_user(user_id=c5810ecbb17a4b1ea3085dd850c6fa2d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:47.198486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068e6e7-fe08-49ec-8866-9cb5c76080df None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:47.200162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 60/119] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:47 2022] GET /identity/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d => generated 275 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:47.200910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068e6e7-fe08-49ec-8866-9cb5c76080df None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:47.269158 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3e73b-769e-46d1-aece-b024a6a9a113 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:47.293825 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068e6e7-fe08-49ec-8866-9cb5c76080df None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:47.301061 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e19dfe-e512-4990-9272-59c7f2dc3b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:47.301703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e19dfe-e512-4990-9272-59c7f2dc3b1d None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:47.301990 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e19dfe-e512-4990-9272-59c7f2dc3b1d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:47.302271 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e19dfe-e512-4990-9272-59c7f2dc3b1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:47.325949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14786ed8-ad1f-4219-8162-b43274535daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:47.326600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14786ed8-ad1f-4219-8162-b43274535daf None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:47.326912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14786ed8-ad1f-4219-8162-b43274535daf None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:47.327189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14786ed8-ad1f-4219-8162-b43274535daf None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:47.327922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14786ed8-ad1f-4219-8162-b43274535daf None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:47.328288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14786ed8-ad1f-4219-8162-b43274535daf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:47.330807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14786ed8-ad1f-4219-8162-b43274535daf None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:47.339898 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 61/120] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:47.345511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6039936-8100-428a-a278-1f13a071a1bd None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:47.345511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6039936-8100-428a-a278-1f13a071a1bd None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:47.345511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6039936-8100-428a-a278-1f13a071a1bd None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:47.345511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 62/121] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:47.345511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6039936-8100-428a-a278-1f13a071a1bd None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:47.377999 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e19dfe-e512-4990-9272-59c7f2dc3b1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:47.379786 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e19dfe-e512-4990-9272-59c7f2dc3b1d None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:47.440470 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6039936-8100-428a-a278-1f13a071a1bd None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:47.470710 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 60/122] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:37:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:37:47.471614 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e19dfe-e512-4990-9272-59c7f2dc3b1d None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:47.472367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7394e33-af13-43ca-8c12-01e6b9a77c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:47.473041 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7394e33-af13-43ca-8c12-01e6b9a77c97 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:47.473334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7394e33-af13-43ca-8c12-01e6b9a77c97 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:47.473616 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7394e33-af13-43ca-8c12-01e6b9a77c97 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:47.474328 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7394e33-af13-43ca-8c12-01e6b9a77c97 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:47.475831 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7394e33-af13-43ca-8c12-01e6b9a77c97 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:47.503795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 63/123] 10.222.0.35 () {60 vars in 1141 bytes} [Fri May 13 21:37:47 2022] POST /identity/v3/projects => generated 302 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:37:47.504613 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7394e33-af13-43ca-8c12-01e6b9a77c97 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:47.584744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e19dfe-e512-4990-9272-59c7f2dc3b1d None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:47.610334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7394e33-af13-43ca-8c12-01e6b9a77c97 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:47.622013 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee52d9-47ef-4b32-8c67-d158bb5572be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:47.623062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee52d9-47ef-4b32-8c67-d158bb5572be None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:47.623350 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee52d9-47ef-4b32-8c67-d158bb5572be None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:47.623628 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee52d9-47ef-4b32-8c67-d158bb5572be None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a/users/c5810ecbb17a4b1ea3085dd850c6fa2d/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:47.626245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee52d9-47ef-4b32-8c67-d158bb5572be None admin] RBAC: Authorizing `identity:create_grant(project_id=80047f4aaba64866971a8b97d21dee0a, user_id=c5810ecbb17a4b1ea3085dd850c6fa2d, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:47.627744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee52d9-47ef-4b32-8c67-d158bb5572be None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:47.642858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 61/124] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:37:47 2022] PUT /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a/users/c5810ecbb17a4b1ea3085dd850c6fa2d/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:37:47.643734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee52d9-47ef-4b32-8c67-d158bb5572be None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:47.647994 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90c2b8-fded-4e91-b620-c2db51f9e958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:47.648877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90c2b8-fded-4e91-b620-c2db51f9e958 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:47.649169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90c2b8-fded-4e91-b620-c2db51f9e958 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:47.649447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90c2b8-fded-4e91-b620-c2db51f9e958 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:47.650775 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90c2b8-fded-4e91-b620-c2db51f9e958 None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:47.652270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90c2b8-fded-4e91-b620-c2db51f9e958 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:47.654247 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 64/125] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:47 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:47.655358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf319627-b73f-4621-ae34-ae8ef27bc22f None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:47.655648 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf319627-b73f-4621-ae34-ae8ef27bc22f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:47.655929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf319627-b73f-4621-ae34-ae8ef27bc22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:47.684052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-bf319627-b73f-4621-ae34-ae8ef27bc22f None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:47.687115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-bf319627-b73f-4621-ae34-ae8ef27bc22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:47.781386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee52d9-47ef-4b32-8c67-d158bb5572be None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:47.789221 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 65/126] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:47.790148 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf319627-b73f-4621-ae34-ae8ef27bc22f None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:47.820380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae31bb-d9ec-4bd0-91a0-e519408ab0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:47.821550 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae31bb-d9ec-4bd0-91a0-e519408ab0df None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:47.821990 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae31bb-d9ec-4bd0-91a0-e519408ab0df None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:47.822453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae31bb-d9ec-4bd0-91a0-e519408ab0df None admin] PATH_INFO: `/v3/users/3b8d43206c7f4426b0be3e6d78690460` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:47.823970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae31bb-d9ec-4bd0-91a0-e519408ab0df None admin] RBAC: Authorizing `identity:get_user(user_id=3b8d43206c7f4426b0be3e6d78690460)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:47.825660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae31bb-d9ec-4bd0-91a0-e519408ab0df None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:47.827687 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 62/127] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:47 2022] GET /identity/v3/users/3b8d43206c7f4426b0be3e6d78690460 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:47.828626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae31bb-d9ec-4bd0-91a0-e519408ab0df None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:47.899912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf319627-b73f-4621-ae34-ae8ef27bc22f None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:47.929618 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae31bb-d9ec-4bd0-91a0-e519408ab0df None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:47.934391 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a91931-2779-485a-8eeb-cd73a0156318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:47.935414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a91931-2779-485a-8eeb-cd73a0156318 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:47.935710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a91931-2779-485a-8eeb-cd73a0156318 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:47.935991 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a91931-2779-485a-8eeb-cd73a0156318 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:47.937203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a91931-2779-485a-8eeb-cd73a0156318 None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:47.939377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a91931-2779-485a-8eeb-cd73a0156318 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:47.941648 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 66/128] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:47 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:47.941920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a91931-2779-485a-8eeb-cd73a0156318 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:47.967306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38a5f0-eb06-4539-a934-cebedcbd8619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:47.968249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38a5f0-eb06-4539-a934-cebedcbd8619 None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:47.968534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38a5f0-eb06-4539-a934-cebedcbd8619 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:47.968897 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38a5f0-eb06-4539-a934-cebedcbd8619 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a/users/c5810ecbb17a4b1ea3085dd850c6fa2d/roles/a29f4364896c46dd84dcee5b969b9898` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:47.971917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f38a5f0-eb06-4539-a934-cebedcbd8619 None admin] RBAC: Authorizing `identity:create_grant(project_id=80047f4aaba64866971a8b97d21dee0a, user_id=c5810ecbb17a4b1ea3085dd850c6fa2d, role_id=a29f4364896c46dd84dcee5b969b9898)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:47.973626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f38a5f0-eb06-4539-a934-cebedcbd8619 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:47.992017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 63/129] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:37:47 2022] PUT /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a/users/c5810ecbb17a4b1ea3085dd850c6fa2d/roles/a29f4364896c46dd84dcee5b969b9898 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:37:47.992842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38a5f0-eb06-4539-a934-cebedcbd8619 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:48.059682 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a91931-2779-485a-8eeb-cd73a0156318 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:48.099326 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aae859-d65c-4ca2-851a-3b661c204910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:48.100391 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aae859-d65c-4ca2-851a-3b661c204910 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:48.100819 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aae859-d65c-4ca2-851a-3b661c204910 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:48.101252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aae859-d65c-4ca2-851a-3b661c204910 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:48.113151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aae859-d65c-4ca2-851a-3b661c204910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-d2aae859-d65c-4ca2-851a-3b661c204910 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:48.121681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:48.122893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:48.123780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:48.123780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:48.123780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:48.123780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:48.123780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:48.123780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:48.123780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:48.123780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:48.123780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:48.123780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:48.123780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:37:48.123780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:37:48.125809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 67/130] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:48.126848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aae859-d65c-4ca2-851a-3b661c204910 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:48.127003 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38a5f0-eb06-4539-a934-cebedcbd8619 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:48.169977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b75ed2-0be1-4973-aec6-730fa5cdccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:48.170896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b75ed2-0be1-4973-aec6-730fa5cdccd8 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:48.171180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b75ed2-0be1-4973-aec6-730fa5cdccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:48.171461 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b75ed2-0be1-4973-aec6-730fa5cdccd8 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:48.173342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b75ed2-0be1-4973-aec6-730fa5cdccd8 None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:48.174882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b75ed2-0be1-4973-aec6-730fa5cdccd8 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:48.177367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 64/131] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:47 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:48.178609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b75ed2-0be1-4973-aec6-730fa5cdccd8 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:48.242240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aae859-d65c-4ca2-851a-3b661c204910 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:48.277659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e461c8-76f6-4ee7-8160-7502155d2057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:48.278787 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e461c8-76f6-4ee7-8160-7502155d2057 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:48.279283 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e461c8-76f6-4ee7-8160-7502155d2057 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:48.279777 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e461c8-76f6-4ee7-8160-7502155d2057 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:48.280836 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e461c8-76f6-4ee7-8160-7502155d2057 None admin] RBAC: Adding query filter params (role.id=94938baf3b7443bd8e53ce11902c7247, scope.project.id=80047f4aaba64866971a8b97d21dee0a, user.id=1984a1910cb74be89a85e92df2957b48) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:48.281978 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e461c8-76f6-4ee7-8160-7502155d2057 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:48.283649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b75ed2-0be1-4973-aec6-730fa5cdccd8 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:48.283995 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e461c8-76f6-4ee7-8160-7502155d2057 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:48.290323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 68/132] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:37:48 2022] GET /identity/v3/role_assignments?user.id=1984a1910cb74be89a85e92df2957b48&scope.project.id=80047f4aaba64866971a8b97d21dee0a&role.id=94938baf3b7443bd8e53ce11902c7247 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:48.291705 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e461c8-76f6-4ee7-8160-7502155d2057 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:48.344828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ec58f-9d3c-47d9-9199-0fe121b1006c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:48.345496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ec58f-9d3c-47d9-9199-0fe121b1006c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:48.345701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ec58f-9d3c-47d9-9199-0fe121b1006c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:48.345959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ec58f-9d3c-47d9-9199-0fe121b1006c None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:48.347433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ec58f-9d3c-47d9-9199-0fe121b1006c None admin] RBAC: Adding query filter params (name=member) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:48.347433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ec58f-9d3c-47d9-9199-0fe121b1006c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:48.348567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ec58f-9d3c-47d9-9199-0fe121b1006c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:48.363986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 65/133] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:37:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:48.369426 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ec58f-9d3c-47d9-9199-0fe121b1006c None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:48.412281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e461c8-76f6-4ee7-8160-7502155d2057 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:48.447373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335ef96-d655-4adf-bbc8-0ea441a3cc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:48.448636 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335ef96-d655-4adf-bbc8-0ea441a3cc6b None admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:48.449148 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335ef96-d655-4adf-bbc8-0ea441a3cc6b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:48.449632 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335ef96-d655-4adf-bbc8-0ea441a3cc6b None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a/users/3b8d43206c7f4426b0be3e6d78690460/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:48.451985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0335ef96-d655-4adf-bbc8-0ea441a3cc6b None admin] RBAC: Authorizing `identity:create_grant(project_id=80047f4aaba64866971a8b97d21dee0a, user_id=3b8d43206c7f4426b0be3e6d78690460, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:48.454315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0335ef96-d655-4adf-bbc8-0ea441a3cc6b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:48.473133 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 69/134] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:37:48 2022] PUT /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a/users/3b8d43206c7f4426b0be3e6d78690460/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:37:48.513399 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ec58f-9d3c-47d9-9199-0fe121b1006c None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:48.554922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada22a5a-6994-415b-8f1a-9c181d460283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:48.556357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada22a5a-6994-415b-8f1a-9c181d460283 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:48.556853 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada22a5a-6994-415b-8f1a-9c181d460283 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:48.557337 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada22a5a-6994-415b-8f1a-9c181d460283 None admin] PATH_INFO: `/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:48.559290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada22a5a-6994-415b-8f1a-9c181d460283 None admin] RBAC: Authorizing `identity:get_user(user_id=c5810ecbb17a4b1ea3085dd850c6fa2d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:48.561044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada22a5a-6994-415b-8f1a-9c181d460283 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:48.563298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 66/135] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:48 2022] GET /identity/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:48.571140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335ef96-d655-4adf-bbc8-0ea441a3cc6b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:48.684083 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335ef96-d655-4adf-bbc8-0ea441a3cc6b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:48.720627 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227507d3-c328-479d-8d51-781e265aa9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:48.721900 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227507d3-c328-479d-8d51-781e265aa9eb None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:48.722417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227507d3-c328-479d-8d51-781e265aa9eb None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:48.722931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227507d3-c328-479d-8d51-781e265aa9eb None admin] PATH_INFO: `/v3/projects/7bf51f2a7fbd437e8dd20b79f52ea901` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:48.724430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227507d3-c328-479d-8d51-781e265aa9eb None admin] RBAC: Authorizing `identity:get_project(project_id=7bf51f2a7fbd437e8dd20b79f52ea901)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:48.726119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227507d3-c328-479d-8d51-781e265aa9eb None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:48.728148 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 70/136] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:48 2022] GET /identity/v3/projects/7bf51f2a7fbd437e8dd20b79f52ea901 => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:48.733972 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada22a5a-6994-415b-8f1a-9c181d460283 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:48.845667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada22a5a-6994-415b-8f1a-9c181d460283 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:48.880689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ad7c1-30a3-453c-93f7-cf660de6d91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:48.881938 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ad7c1-30a3-453c-93f7-cf660de6d91d None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:48.882420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ad7c1-30a3-453c-93f7-cf660de6d91d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:48.882943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ad7c1-30a3-453c-93f7-cf660de6d91d None admin] PATH_INFO: `/v3/projects/7bf51f2a7fbd437e8dd20b79f52ea901/users/c5810ecbb17a4b1ea3085dd850c6fa2d/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:48.885181 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ad7c1-30a3-453c-93f7-cf660de6d91d None admin] RBAC: Authorizing `identity:create_grant(project_id=7bf51f2a7fbd437e8dd20b79f52ea901, user_id=c5810ecbb17a4b1ea3085dd850c6fa2d, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:48.886901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ad7c1-30a3-453c-93f7-cf660de6d91d None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:48.915609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 67/137] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:37:48 2022] PUT /identity/v3/projects/7bf51f2a7fbd437e8dd20b79f52ea901/users/c5810ecbb17a4b1ea3085dd850c6fa2d/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:37:50.600773 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f2fb9-ec77-4692-9f8a-b00057429359 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:50.601288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f2fb9-ec77-4692-9f8a-b00057429359 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:50.601362 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f2fb9-ec77-4692-9f8a-b00057429359 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:50.602690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 71/138] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:50.607908 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8bb3b-e911-4893-9ea1-4537e6b4b8f8 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:50.608319 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8bb3b-e911-4893-9ea1-4537e6b4b8f8 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:50.608485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8bb3b-e911-4893-9ea1-4537e6b4b8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:50.630644 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-06e8bb3b-e911-4893-9ea1-4537e6b4b8f8 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:50.631869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-06e8bb3b-e911-4893-9ea1-4537e6b4b8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:50.717267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 68/139] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:50.752370 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f2fb9-ec77-4692-9f8a-b00057429359 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:50.860869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f2fb9-ec77-4692-9f8a-b00057429359 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:50.888602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e22381-5a5c-4c09-beb7-aac2d29f1230 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:50.888908 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e22381-5a5c-4c09-beb7-aac2d29f1230 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:50.889630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e22381-5a5c-4c09-beb7-aac2d29f1230 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:50.896905 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 69/140] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:50.899525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dada2-fca3-4b75-99ec-f3e89d474265 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:50.899816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dada2-fca3-4b75-99ec-f3e89d474265 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:50.900440 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dada2-fca3-4b75-99ec-f3e89d474265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:50.916353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ac368-ec62-40a3-8538-dae16dcad552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:50.917162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ac368-ec62-40a3-8538-dae16dcad552 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:50.917448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ac368-ec62-40a3-8538-dae16dcad552 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:50.917729 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ac368-ec62-40a3-8538-dae16dcad552 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:50.918451 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-f64dada2-fca3-4b75-99ec-f3e89d474265 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:50.919732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f64dada2-fca3-4b75-99ec-f3e89d474265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:50.944126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ac368-ec62-40a3-8538-dae16dcad552 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-f13ac368-ec62-40a3-8538-dae16dcad552 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:50.951827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:50.952680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:50.954597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:50.954597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:50.954597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:50.954597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:50.954597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:50.954597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:50.954597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:50.954597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:50.954597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:50.954597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:50.954597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:37:50.954597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:37:50.954597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 72/141] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:50.959277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ac368-ec62-40a3-8538-dae16dcad552 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:50.992994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 70/142] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:37:51.065284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64dada2-fca3-4b75-99ec-f3e89d474265 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:51.101622 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ac368-ec62-40a3-8538-dae16dcad552 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:51.177355 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cae50b-8eb1-47b7-89f2-e7e23107ffb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:51.178378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cae50b-8eb1-47b7-89f2-e7e23107ffb8 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:51.178895 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cae50b-8eb1-47b7-89f2-e7e23107ffb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:51.179380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cae50b-8eb1-47b7-89f2-e7e23107ffb8 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:51.180330 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cae50b-8eb1-47b7-89f2-e7e23107ffb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:51.180872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cae50b-8eb1-47b7-89f2-e7e23107ffb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:51.182483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cae50b-8eb1-47b7-89f2-e7e23107ffb8 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:51.193721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64dada2-fca3-4b75-99ec-f3e89d474265 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:51.205773 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 73/143] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:37:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:51.210776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cae50b-8eb1-47b7-89f2-e7e23107ffb8 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:51.231230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6a145-53a5-4e15-a096-64b86bfce42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:51.232245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6a145-53a5-4e15-a096-64b86bfce42d None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:51.232516 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6a145-53a5-4e15-a096-64b86bfce42d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:51.232816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6a145-53a5-4e15-a096-64b86bfce42d None admin] PATH_INFO: `/v3/domains/default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:51.234216 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6a145-53a5-4e15-a096-64b86bfce42d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:51.236030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6a145-53a5-4e15-a096-64b86bfce42d None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:51.238382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 71/144] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:37:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:51.247752 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6a145-53a5-4e15-a096-64b86bfce42d None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:51.330023 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cae50b-8eb1-47b7-89f2-e7e23107ffb8 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:51.353945 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6a145-53a5-4e15-a096-64b86bfce42d None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:51.364807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bdf99-e9f4-4125-bbb7-6a1ea9f5d15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:51.366040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bdf99-e9f4-4125-bbb7-6a1ea9f5d15e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:51.366547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bdf99-e9f4-4125-bbb7-6a1ea9f5d15e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:51.367055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bdf99-e9f4-4125-bbb7-6a1ea9f5d15e None admin] PATH_INFO: `/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:51.368495 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bdf99-e9f4-4125-bbb7-6a1ea9f5d15e None admin] RBAC: Authorizing `identity:get_user(user_id=c5810ecbb17a4b1ea3085dd850c6fa2d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:51.370185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bdf99-e9f4-4125-bbb7-6a1ea9f5d15e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:51.372183 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 74/145] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:51 2022] GET /identity/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:51.379560 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bdf99-e9f4-4125-bbb7-6a1ea9f5d15e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:51.396737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d0f43-f04a-4f6c-8717-7383ae122069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:51.397421 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d0f43-f04a-4f6c-8717-7383ae122069 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:51.397712 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d0f43-f04a-4f6c-8717-7383ae122069 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:51.397978 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d0f43-f04a-4f6c-8717-7383ae122069 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:51.398686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d0f43-f04a-4f6c-8717-7383ae122069 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:51.400148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d0f43-f04a-4f6c-8717-7383ae122069 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:51.454751 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 72/146] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:37:51 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:37:51.496086 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bdf99-e9f4-4125-bbb7-6a1ea9f5d15e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:51.514429 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033efba0-52d5-41bf-9b6c-e8424426de27 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:51.514765 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033efba0-52d5-41bf-9b6c-e8424426de27 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:51.515026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033efba0-52d5-41bf-9b6c-e8424426de27 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:51.516768 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 73/147] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:37:51.524459 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdfaec4-7702-4a4f-831d-a45ffd2f7e24 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:51.524459 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdfaec4-7702-4a4f-831d-a45ffd2f7e24 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:51.524807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdfaec4-7702-4a4f-831d-a45ffd2f7e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:51.549806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b314e4-581e-4eab-8965-cb51e26dc5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:51.550480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b314e4-581e-4eab-8965-cb51e26dc5b7 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:51.550824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b314e4-581e-4eab-8965-cb51e26dc5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:51.551169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b314e4-581e-4eab-8965-cb51e26dc5b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:51.560119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-8cdfaec4-7702-4a4f-831d-a45ffd2f7e24 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:51.561162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-8cdfaec4-7702-4a4f-831d-a45ffd2f7e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:51.672661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 74/148] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:51.688073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b314e4-581e-4eab-8965-cb51e26dc5b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:51.689609 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b314e4-581e-4eab-8965-cb51e26dc5b7 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:51.707093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdfaec4-7702-4a4f-831d-a45ffd2f7e24 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:51.823491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdfaec4-7702-4a4f-831d-a45ffd2f7e24 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:51.834845 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 75/149] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:37:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:37:51.836470 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c72e-b536-478b-a14b-a8bf5e9a4e91 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:51.836696 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c72e-b536-478b-a14b-a8bf5e9a4e91 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:51.836954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c72e-b536-478b-a14b-a8bf5e9a4e91 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:51.838351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 76/150] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:51.840627 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762c72e-b536-478b-a14b-a8bf5e9a4e91 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:51.861610 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89525971-796f-466f-9f2f-6c1d06417faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:51.862415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89525971-796f-466f-9f2f-6c1d06417faa None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:51.862720 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89525971-796f-466f-9f2f-6c1d06417faa None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:51.863002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89525971-796f-466f-9f2f-6c1d06417faa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:51.873840 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89525971-796f-466f-9f2f-6c1d06417faa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-89525971-796f-466f-9f2f-6c1d06417faa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:51.882288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:51.883201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:51.883961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:51.883961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:51.883961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:51.883961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:51.883961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:51.883961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:51.883961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:51.883961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:51.883961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:51.883961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:51.883961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:37:51.883961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:37:51.883961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 75/151] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:37:51.885180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4d635-4bc1-40c7-abc0-c687f1e9f901 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:51.885474 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4d635-4bc1-40c7-abc0-c687f1e9f901 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:51.885757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4d635-4bc1-40c7-abc0-c687f1e9f901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:51.904180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d7a4d635-4bc1-40c7-abc0-c687f1e9f901 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:51.905361 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d7a4d635-4bc1-40c7-abc0-c687f1e9f901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:51.959729 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762c72e-b536-478b-a14b-a8bf5e9a4e91 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:51.974862 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 76/152] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:51.975856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4d635-4bc1-40c7-abc0-c687f1e9f901 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:51.996464 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ac0de-8d98-4970-a4bd-8b8f1161f983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:51.997748 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ac0de-8d98-4970-a4bd-8b8f1161f983 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:51.998265 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ac0de-8d98-4970-a4bd-8b8f1161f983 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:51.998779 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ac0de-8d98-4970-a4bd-8b8f1161f983 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:52.000801 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ac0de-8d98-4970-a4bd-8b8f1161f983 None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:52.002670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ac0de-8d98-4970-a4bd-8b8f1161f983 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:52.005281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 77/153] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:51 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:52.010166 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ac0de-8d98-4970-a4bd-8b8f1161f983 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:52.075841 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4d635-4bc1-40c7-abc0-c687f1e9f901 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:52.110803 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f408ad-1050-4cce-880d-7bb7f7f4b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:52.111480 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f408ad-1050-4cce-880d-7bb7f7f4b6cc None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:52.111760 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f408ad-1050-4cce-880d-7bb7f7f4b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:52.112095 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f408ad-1050-4cce-880d-7bb7f7f4b6cc None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:52.112846 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f408ad-1050-4cce-880d-7bb7f7f4b6cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:52.113221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f408ad-1050-4cce-880d-7bb7f7f4b6cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:52.114534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f408ad-1050-4cce-880d-7bb7f7f4b6cc None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:52.122824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ac0de-8d98-4970-a4bd-8b8f1161f983 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:52.124505 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 77/154] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:37:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:52.125937 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f408ad-1050-4cce-880d-7bb7f7f4b6cc None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:52.165998 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b40312-5655-4480-a37b-c4d87984dc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:52.167204 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b40312-5655-4480-a37b-c4d87984dc89 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:52.167723 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b40312-5655-4480-a37b-c4d87984dc89 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:52.168198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b40312-5655-4480-a37b-c4d87984dc89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:52.179407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b40312-5655-4480-a37b-c4d87984dc89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-34b40312-5655-4480-a37b-c4d87984dc89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:52.194073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:52.195019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:52.195772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:52.195772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:52.195772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:52.195772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:52.195772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:52.195772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:52.195772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:52.195772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:52.195772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:52.195772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:52.195772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:37:52.195772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:37:52.199007 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 78/155] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:37:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:52.200117 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b40312-5655-4480-a37b-c4d87984dc89 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:52.233081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f408ad-1050-4cce-880d-7bb7f7f4b6cc None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:52.270119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd6f52-b71f-49dc-b5db-c38fdb900f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:52.270804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd6f52-b71f-49dc-b5db-c38fdb900f4a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:52.271089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd6f52-b71f-49dc-b5db-c38fdb900f4a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:52.271367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd6f52-b71f-49dc-b5db-c38fdb900f4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:52.272204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd6f52-b71f-49dc-b5db-c38fdb900f4a None admin] RBAC: Adding query filter params (role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d, scope.project.id=80047f4aaba64866971a8b97d21dee0a, user.id=c5810ecbb17a4b1ea3085dd850c6fa2d) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:52.272515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd6f52-b71f-49dc-b5db-c38fdb900f4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:52.273961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd6f52-b71f-49dc-b5db-c38fdb900f4a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:52.281243 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 78/156] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:37:52 2022] GET /identity/v3/role_assignments?user.id=c5810ecbb17a4b1ea3085dd850c6fa2d&scope.project.id=80047f4aaba64866971a8b97d21dee0a&role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:52.282072 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd6f52-b71f-49dc-b5db-c38fdb900f4a None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:52.314099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b40312-5655-4480-a37b-c4d87984dc89 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:52.349924 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90a530-b796-44d7-81cb-9ed00a483614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:52.351285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90a530-b796-44d7-81cb-9ed00a483614 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:52.351826 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90a530-b796-44d7-81cb-9ed00a483614 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:52.352318 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90a530-b796-44d7-81cb-9ed00a483614 None admin] PATH_INFO: `/v3/users/1984a1910cb74be89a85e92df2957b48` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:52.353919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90a530-b796-44d7-81cb-9ed00a483614 None admin] RBAC: Authorizing `identity:get_user(user_id=1984a1910cb74be89a85e92df2957b48)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:52.355765 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90a530-b796-44d7-81cb-9ed00a483614 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:52.358531 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 79/157] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:52 2022] GET /identity/v3/users/1984a1910cb74be89a85e92df2957b48 => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:52.364140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90a530-b796-44d7-81cb-9ed00a483614 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:52.396176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd6f52-b71f-49dc-b5db-c38fdb900f4a None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:52.466655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d91be4-6c46-47b4-b331-58a5a2db84fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:52.467782 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d91be4-6c46-47b4-b331-58a5a2db84fc None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:52.468272 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d91be4-6c46-47b4-b331-58a5a2db84fc None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:52.469414 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d91be4-6c46-47b4-b331-58a5a2db84fc None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:52.472096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d91be4-6c46-47b4-b331-58a5a2db84fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:52.472679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d91be4-6c46-47b4-b331-58a5a2db84fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:52.474429 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d91be4-6c46-47b4-b331-58a5a2db84fc None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:52.483898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 79/158] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:37:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:52.485589 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1a0af-2b81-43fa-b7fa-d51500b768b2 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:52.486120 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1a0af-2b81-43fa-b7fa-d51500b768b2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:52.486608 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1a0af-2b81-43fa-b7fa-d51500b768b2 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:52.488168 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 80/159] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:52.489222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1a0af-2b81-43fa-b7fa-d51500b768b2 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:52.508465 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90a530-b796-44d7-81cb-9ed00a483614 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:52.545264 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ccd51-f808-4204-88d0-7ba1da075049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:52.546200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ccd51-f808-4204-88d0-7ba1da075049 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:52.546518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ccd51-f808-4204-88d0-7ba1da075049 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:52.546887 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ccd51-f808-4204-88d0-7ba1da075049 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:52.548212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ccd51-f808-4204-88d0-7ba1da075049 None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:52.549762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ccd51-f808-4204-88d0-7ba1da075049 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:52.551806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 80/160] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:52 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:52.553013 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837dd78-e134-46ed-ba4e-c2fb7fa2c241 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:52.553307 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837dd78-e134-46ed-ba4e-c2fb7fa2c241 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:52.553585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837dd78-e134-46ed-ba4e-c2fb7fa2c241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:52.577149 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-3837dd78-e134-46ed-ba4e-c2fb7fa2c241 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:52.578976 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-3837dd78-e134-46ed-ba4e-c2fb7fa2c241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:52.622213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1a0af-2b81-43fa-b7fa-d51500b768b2 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:52.655151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 81/161] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:52.657304 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8a8f3-5e47-4a84-a95f-43ef8e31aed0 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:52.657858 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8a8f3-5e47-4a84-a95f-43ef8e31aed0 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:52.658452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8a8f3-5e47-4a84-a95f-43ef8e31aed0 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:52.658696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decc53e-40ea-42a1-8eee-13fb4d14ff83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:52.659707 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decc53e-40ea-42a1-8eee-13fb4d14ff83 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:52.660057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decc53e-40ea-42a1-8eee-13fb4d14ff83 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:52.660391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decc53e-40ea-42a1-8eee-13fb4d14ff83 None admin] PATH_INFO: `/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:52.661242 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 82/162] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:52.661748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decc53e-40ea-42a1-8eee-13fb4d14ff83 None admin] RBAC: Authorizing `identity:get_user(user_id=c5810ecbb17a4b1ea3085dd850c6fa2d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:52.662403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8a8f3-5e47-4a84-a95f-43ef8e31aed0 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:52.667852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decc53e-40ea-42a1-8eee-13fb4d14ff83 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:52.669934 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 81/163] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:52 2022] GET /identity/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:52.671234 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43c8f7-838b-40df-b3f2-1c18a6de46b5 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:52.671592 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43c8f7-838b-40df-b3f2-1c18a6de46b5 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:52.671984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43c8f7-838b-40df-b3f2-1c18a6de46b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:52.690039 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-cf43c8f7-838b-40df-b3f2-1c18a6de46b5 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:52.691334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-cf43c8f7-838b-40df-b3f2-1c18a6de46b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:52.789441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 82/164] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:52.792085 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43c8f7-838b-40df-b3f2-1c18a6de46b5 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:52.817496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8a8f3-5e47-4a84-a95f-43ef8e31aed0 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:52.857941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4eca40-9df4-4337-89bf-c17ffb82d408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:52.858916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4eca40-9df4-4337-89bf-c17ffb82d408 None admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:52.859194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4eca40-9df4-4337-89bf-c17ffb82d408 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:52.859495 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4eca40-9df4-4337-89bf-c17ffb82d408 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a/users/1984a1910cb74be89a85e92df2957b48/roles/94938baf3b7443bd8e53ce11902c7247` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:52.861643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4eca40-9df4-4337-89bf-c17ffb82d408 None admin] RBAC: Authorizing `identity:create_grant(project_id=80047f4aaba64866971a8b97d21dee0a, user_id=1984a1910cb74be89a85e92df2957b48, role_id=94938baf3b7443bd8e53ce11902c7247)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:52.863193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4eca40-9df4-4337-89bf-c17ffb82d408 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:52.880483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 83/165] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:37:52 2022] PUT /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a/users/1984a1910cb74be89a85e92df2957b48/roles/94938baf3b7443bd8e53ce11902c7247 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:37:52.881191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4eca40-9df4-4337-89bf-c17ffb82d408 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:52.912333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43c8f7-838b-40df-b3f2-1c18a6de46b5 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:53.002084 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d0cc4-a136-44cc-ab45-a8a378232fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:53.002840 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d0cc4-a136-44cc-ab45-a8a378232fad None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:53.003127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d0cc4-a136-44cc-ab45-a8a378232fad None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:53.003660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d0cc4-a136-44cc-ab45-a8a378232fad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:53.152777 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4eca40-9df4-4337-89bf-c17ffb82d408 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:53.159964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d0cc4-a136-44cc-ab45-a8a378232fad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:53.161482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d0cc4-a136-44cc-ab45-a8a378232fad None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:53.231048 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23ba74-19a3-46b5-9fec-a6eebd2a6004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:53.232017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23ba74-19a3-46b5-9fec-a6eebd2a6004 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:53.232287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23ba74-19a3-46b5-9fec-a6eebd2a6004 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:53.232558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23ba74-19a3-46b5-9fec-a6eebd2a6004 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:53.253658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23ba74-19a3-46b5-9fec-a6eebd2a6004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-8f23ba74-19a3-46b5-9fec-a6eebd2a6004 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:53.262219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:53.263196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:53.263954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:53.263954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:53.263954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:53.263954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:53.263954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:53.263954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:53.263954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:53.263954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:53.263954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:53.263954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:53.263954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:37:53.263954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:37:53.263954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 84/166] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:37:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:53.264624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23ba74-19a3-46b5-9fec-a6eebd2a6004 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:53.334346 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 83/167] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:37:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:37:53.335058 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d0cc4-a136-44cc-ab45-a8a378232fad None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:53.452736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23ba74-19a3-46b5-9fec-a6eebd2a6004 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:53.515302 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac78f37-3bba-4276-87bd-515fa7dcb8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:53.516148 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac78f37-3bba-4276-87bd-515fa7dcb8a1 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:53.516432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac78f37-3bba-4276-87bd-515fa7dcb8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:53.516710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac78f37-3bba-4276-87bd-515fa7dcb8a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:53.522081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d0cc4-a136-44cc-ab45-a8a378232fad None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:53.544440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac78f37-3bba-4276-87bd-515fa7dcb8a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-3ac78f37-3bba-4276-87bd-515fa7dcb8a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:53.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:53.559473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:53.560224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:53.560224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:53.560224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:53.560224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:53.560224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:53.560224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:53.560224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:53.560224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:53.560224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:53.560224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:53.560224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:37:53.560224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:37:53.560224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 85/168] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:53.565698 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac78f37-3bba-4276-87bd-515fa7dcb8a1 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:53.597305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53293610-e12a-4366-9a3d-2335b25005df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:53.598318 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53293610-e12a-4366-9a3d-2335b25005df None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:53.598811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53293610-e12a-4366-9a3d-2335b25005df None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:53.599304 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53293610-e12a-4366-9a3d-2335b25005df None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:53.600237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53293610-e12a-4366-9a3d-2335b25005df None admin] RBAC: Adding query filter params (name=admin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:53.600748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53293610-e12a-4366-9a3d-2335b25005df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:53.602310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53293610-e12a-4366-9a3d-2335b25005df None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:53.621838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 84/169] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:53.622967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53293610-e12a-4366-9a3d-2335b25005df None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:53.686618 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac78f37-3bba-4276-87bd-515fa7dcb8a1 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:53.722815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8865e-6352-43a4-bba9-f35d463f4a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:53.722815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8865e-6352-43a4-bba9-f35d463f4a36 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:53.722815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8865e-6352-43a4-bba9-f35d463f4a36 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:53.722815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8865e-6352-43a4-bba9-f35d463f4a36 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:53.723384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8865e-6352-43a4-bba9-f35d463f4a36 None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:53.724912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8865e-6352-43a4-bba9-f35d463f4a36 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:53.727100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 86/170] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:53 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:53.727944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8865e-6352-43a4-bba9-f35d463f4a36 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:53.780866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53293610-e12a-4366-9a3d-2335b25005df None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:53.833785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8865e-6352-43a4-bba9-f35d463f4a36 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:53.839968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452491d6-7dcb-43f9-88e1-94ee8942a4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:53.840932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452491d6-7dcb-43f9-88e1-94ee8942a4ec None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:53.841424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452491d6-7dcb-43f9-88e1-94ee8942a4ec None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:53.841888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452491d6-7dcb-43f9-88e1-94ee8942a4ec None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:53.842854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452491d6-7dcb-43f9-88e1-94ee8942a4ec None admin] RBAC: Adding query filter params (name=member) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:53.843418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452491d6-7dcb-43f9-88e1-94ee8942a4ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:53.845574 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452491d6-7dcb-43f9-88e1-94ee8942a4ec None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:53.863164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 85/171] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:37:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:53.864315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452491d6-7dcb-43f9-88e1-94ee8942a4ec None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:53.869114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498488a-5f55-4233-81b3-151be36c030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:53.869999 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498488a-5f55-4233-81b3-151be36c030f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:53.870282 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498488a-5f55-4233-81b3-151be36c030f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:53.870556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498488a-5f55-4233-81b3-151be36c030f None admin] PATH_INFO: `/v3/users/3b8d43206c7f4426b0be3e6d78690460` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:53.872011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498488a-5f55-4233-81b3-151be36c030f None admin] RBAC: Authorizing `identity:get_user(user_id=3b8d43206c7f4426b0be3e6d78690460)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:53.873617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498488a-5f55-4233-81b3-151be36c030f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:53.875375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 87/172] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:53 2022] GET /identity/v3/users/3b8d43206c7f4426b0be3e6d78690460 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:53.876145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498488a-5f55-4233-81b3-151be36c030f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:53.979116 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498488a-5f55-4233-81b3-151be36c030f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:54.012266 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc1183-4e75-4277-af43-fae32f0c2881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:54.013136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc1183-4e75-4277-af43-fae32f0c2881 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:54.013417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc1183-4e75-4277-af43-fae32f0c2881 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:54.013692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc1183-4e75-4277-af43-fae32f0c2881 None admin] PATH_INFO: `/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:54.014917 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc1183-4e75-4277-af43-fae32f0c2881 None admin] RBAC: Authorizing `identity:get_user(user_id=c5810ecbb17a4b1ea3085dd850c6fa2d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:54.016915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc1183-4e75-4277-af43-fae32f0c2881 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:54.018745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 88/173] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:53 2022] GET /identity/v3/users/c5810ecbb17a4b1ea3085dd850c6fa2d => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:54.019339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc1183-4e75-4277-af43-fae32f0c2881 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:54.034045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452491d6-7dcb-43f9-88e1-94ee8942a4ec None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:54.104506 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc86f4d-69f0-435e-ad82-a032ff0e451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:54.105164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc86f4d-69f0-435e-ad82-a032ff0e451e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:54.105446 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc86f4d-69f0-435e-ad82-a032ff0e451e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:54.105721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc86f4d-69f0-435e-ad82-a032ff0e451e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:54.106555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc86f4d-69f0-435e-ad82-a032ff0e451e None admin] RBAC: Adding query filter params (role.id=a29f4364896c46dd84dcee5b969b9898, scope.project.id=80047f4aaba64866971a8b97d21dee0a, user.id=c5810ecbb17a4b1ea3085dd850c6fa2d) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:54.106892 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc86f4d-69f0-435e-ad82-a032ff0e451e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:54.108363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc86f4d-69f0-435e-ad82-a032ff0e451e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:54.117609 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc1183-4e75-4277-af43-fae32f0c2881 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:54.123151 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 86/174] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:37:53 2022] GET /identity/v3/role_assignments?user.id=c5810ecbb17a4b1ea3085dd850c6fa2d&scope.project.id=80047f4aaba64866971a8b97d21dee0a&role.id=a29f4364896c46dd84dcee5b969b9898 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:54.123967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc86f4d-69f0-435e-ad82-a032ff0e451e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:54.156507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661ff95-1e39-45ff-bea6-dc26349b850c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:54.157174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661ff95-1e39-45ff-bea6-dc26349b850c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:54.157456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661ff95-1e39-45ff-bea6-dc26349b850c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:54.157731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661ff95-1e39-45ff-bea6-dc26349b850c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:54.232008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661ff95-1e39-45ff-bea6-dc26349b850c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:54.233825 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661ff95-1e39-45ff-bea6-dc26349b850c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:54.341780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc86f4d-69f0-435e-ad82-a032ff0e451e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:54.388961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4c7d0-bf98-4121-aef4-7b919ae02023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:54.390002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4c7d0-bf98-4121-aef4-7b919ae02023 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:54.390551 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4c7d0-bf98-4121-aef4-7b919ae02023 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:54.391554 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4c7d0-bf98-4121-aef4-7b919ae02023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:54.407953 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 89/175] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:37:54.413497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661ff95-1e39-45ff-bea6-dc26349b850c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:54.511563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4c7d0-bf98-4121-aef4-7b919ae02023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:54.513397 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4c7d0-bf98-4121-aef4-7b919ae02023 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:54.555455 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661ff95-1e39-45ff-bea6-dc26349b850c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:54.598206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d834b8-2b51-45f2-96e1-da86efd3fb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:54.599717 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d834b8-2b51-45f2-96e1-da86efd3fb51 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:54.599934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d834b8-2b51-45f2-96e1-da86efd3fb51 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:54.600211 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d834b8-2b51-45f2-96e1-da86efd3fb51 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:54.601439 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d834b8-2b51-45f2-96e1-da86efd3fb51 None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:54.602947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d834b8-2b51-45f2-96e1-da86efd3fb51 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:54.604685 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 90/176] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:54 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:54.610888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d834b8-2b51-45f2-96e1-da86efd3fb51 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:54.618130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 87/177] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:37:54.627684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4c7d0-bf98-4121-aef4-7b919ae02023 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:54.711255 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d834b8-2b51-45f2-96e1-da86efd3fb51 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:54.745932 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14362b28-d1d1-436b-816b-b00eafe267f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:54.746924 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14362b28-d1d1-436b-816b-b00eafe267f0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:54.747042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14362b28-d1d1-436b-816b-b00eafe267f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:54.747227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14362b28-d1d1-436b-816b-b00eafe267f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:54.748114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14362b28-d1d1-436b-816b-b00eafe267f0 None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=80047f4aaba64866971a8b97d21dee0a, user.id=3b8d43206c7f4426b0be3e6d78690460) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:54.748366 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14362b28-d1d1-436b-816b-b00eafe267f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:54.749820 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14362b28-d1d1-436b-816b-b00eafe267f0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:54.750965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4c7d0-bf98-4121-aef4-7b919ae02023 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:54.757147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 91/178] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:37:54 2022] GET /identity/v3/role_assignments?user.id=3b8d43206c7f4426b0be3e6d78690460&scope.project.id=80047f4aaba64866971a8b97d21dee0a&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:54.789099 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1669d8-b3bb-45c0-bf67-91e529bdb1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:54.790690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1669d8-b3bb-45c0-bf67-91e529bdb1ed None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:54.791275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1669d8-b3bb-45c0-bf67-91e529bdb1ed None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:54.791691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1669d8-b3bb-45c0-bf67-91e529bdb1ed None admin] PATH_INFO: `/v3/projects/7bf51f2a7fbd437e8dd20b79f52ea901` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:54.793309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1669d8-b3bb-45c0-bf67-91e529bdb1ed None admin] RBAC: Authorizing `identity:get_project(project_id=7bf51f2a7fbd437e8dd20b79f52ea901)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:54.795034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1669d8-b3bb-45c0-bf67-91e529bdb1ed None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:54.797638 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 88/179] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:54 2022] GET /identity/v3/projects/7bf51f2a7fbd437e8dd20b79f52ea901 => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:54.803874 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14362b28-d1d1-436b-816b-b00eafe267f0 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:54.939957 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14362b28-d1d1-436b-816b-b00eafe267f0 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:54.980288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d44c5-3ca2-4dda-af14-495868096a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:54.980949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d44c5-3ca2-4dda-af14-495868096a62 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:54.981235 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d44c5-3ca2-4dda-af14-495868096a62 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:54.981514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d44c5-3ca2-4dda-af14-495868096a62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:54.982352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d44c5-3ca2-4dda-af14-495868096a62 None admin] RBAC: Adding query filter params (role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d, scope.project.id=7bf51f2a7fbd437e8dd20b79f52ea901, user.id=c5810ecbb17a4b1ea3085dd850c6fa2d) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:54.982666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d44c5-3ca2-4dda-af14-495868096a62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:54.985055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d44c5-3ca2-4dda-af14-495868096a62 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:54.991860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 92/180] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:37:54 2022] GET /identity/v3/role_assignments?user.id=c5810ecbb17a4b1ea3085dd850c6fa2d&scope.project.id=7bf51f2a7fbd437e8dd20b79f52ea901&role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:55.723425 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d3905-2a76-46d0-9b43-dfdb886f8c57 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:55.724127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d3905-2a76-46d0-9b43-dfdb886f8c57 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:55.724636 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d3905-2a76-46d0-9b43-dfdb886f8c57 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:55.726335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 89/181] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:55.732990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a012fb-756f-4c28-b775-5427c10d930b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:55.733297 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a012fb-756f-4c28-b775-5427c10d930b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:55.733553 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a012fb-756f-4c28-b775-5427c10d930b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:55.753888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-73a012fb-756f-4c28-b775-5427c10d930b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:55.755185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-73a012fb-756f-4c28-b775-5427c10d930b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:55.825795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 93/182] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:37:55.849320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d3905-2a76-46d0-9b43-dfdb886f8c57 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:55.890522 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91780f57-a395-49fa-b024-83a9ad79dc9e None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:55.890865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91780f57-a395-49fa-b024-83a9ad79dc9e None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:55.891120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91780f57-a395-49fa-b024-83a9ad79dc9e None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:55.892863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 94/183] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:55.899921 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1d818-7443-4ea5-b844-03a030cfc31e None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:55.900196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1d818-7443-4ea5-b844-03a030cfc31e None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:55.900595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1d818-7443-4ea5-b844-03a030cfc31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:55.917539 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-edf1d818-7443-4ea5-b844-03a030cfc31e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:55.918892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-edf1d818-7443-4ea5-b844-03a030cfc31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:55.959924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d3905-2a76-46d0-9b43-dfdb886f8c57 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:55.991758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 95/184] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:37:55.994094 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291288bf-fdbf-4f5b-8065-b0196847c64b None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:55.994602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291288bf-fdbf-4f5b-8065-b0196847c64b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:55.995096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291288bf-fdbf-4f5b-8065-b0196847c64b None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:55.996558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 96/185] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:55.999427 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04254a88-4056-4619-abf4-d67157e010a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:56.000488 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04254a88-4056-4619-abf4-d67157e010a1 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:56.000980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04254a88-4056-4619-abf4-d67157e010a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:56.001446 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04254a88-4056-4619-abf4-d67157e010a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:56.002120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70715d76-fcc0-492d-86af-10a11b16deca None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:56.002374 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70715d76-fcc0-492d-86af-10a11b16deca None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:56.002684 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70715d76-fcc0-492d-86af-10a11b16deca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:56.012893 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04254a88-4056-4619-abf4-d67157e010a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:56.019476 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-70715d76-fcc0-492d-86af-10a11b16deca None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:56.020650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-70715d76-fcc0-492d-86af-10a11b16deca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-04254a88-4056-4619-abf4-d67157e010a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:56.021602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:56.022434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:56.023227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:56.023227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:56.023227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:56.023227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:56.023227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:56.023227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:56.023227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:56.023227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:56.023227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:56.023227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:56.023227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:37:56.023227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:37:56.027542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 90/186] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:37:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:56.028172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04254a88-4056-4619-abf4-d67157e010a1 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:56.088826 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 97/187] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:56.099582 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70715d76-fcc0-492d-86af-10a11b16deca None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:56.124316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04254a88-4056-4619-abf4-d67157e010a1 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:56.157363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849649c6-2a57-42b6-a8c6-55b9eb69d606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:56.158149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849649c6-2a57-42b6-a8c6-55b9eb69d606 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:56.158430 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849649c6-2a57-42b6-a8c6-55b9eb69d606 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:56.158733 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849649c6-2a57-42b6-a8c6-55b9eb69d606 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:56.159898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849649c6-2a57-42b6-a8c6-55b9eb69d606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:56.161381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849649c6-2a57-42b6-a8c6-55b9eb69d606 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:56.163033 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 91/188] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:37:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:56.163793 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849649c6-2a57-42b6-a8c6-55b9eb69d606 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:56.205725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70715d76-fcc0-492d-86af-10a11b16deca None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:56.237751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2090714-fa14-4740-89cb-4bd22ffeb44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:56.238414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2090714-fa14-4740-89cb-4bd22ffeb44b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:56.238715 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2090714-fa14-4740-89cb-4bd22ffeb44b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:56.239018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2090714-fa14-4740-89cb-4bd22ffeb44b None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:56.239760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2090714-fa14-4740-89cb-4bd22ffeb44b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:56.240070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2090714-fa14-4740-89cb-4bd22ffeb44b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:56.241451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2090714-fa14-4740-89cb-4bd22ffeb44b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:56.249315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 98/189] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:56.250547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4c1c7-212b-41de-a945-5b5bdf178d55 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:56.250879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4c1c7-212b-41de-a945-5b5bdf178d55 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:56.251162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4c1c7-212b-41de-a945-5b5bdf178d55 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:56.252404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 99/190] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:56.253417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4c1c7-212b-41de-a945-5b5bdf178d55 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:56.257077 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849649c6-2a57-42b6-a8c6-55b9eb69d606 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:56.290204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664d8a4-f533-4dbc-a399-ae7a70963c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:56.291047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664d8a4-f533-4dbc-a399-ae7a70963c11 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:56.291330 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664d8a4-f533-4dbc-a399-ae7a70963c11 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:56.291602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664d8a4-f533-4dbc-a399-ae7a70963c11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:56.301356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3664d8a4-f533-4dbc-a399-ae7a70963c11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-3664d8a4-f533-4dbc-a399-ae7a70963c11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:56.309250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:56.310161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:56.310976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:56.310976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:56.310976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:56.310976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:56.310976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:56.310976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:56.310976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:56.310976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:56.310976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:56.310976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:56.310976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:37:56.310976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:37:56.310976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 92/191] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:37:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:56.311602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664d8a4-f533-4dbc-a399-ae7a70963c11 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:56.354120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4c1c7-212b-41de-a945-5b5bdf178d55 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:56.389246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f584e-4f01-49a6-8017-fbc0a10e4fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:56.389914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f584e-4f01-49a6-8017-fbc0a10e4fc7 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:56.390197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f584e-4f01-49a6-8017-fbc0a10e4fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:56.391096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f584e-4f01-49a6-8017-fbc0a10e4fc7 None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:56.391808 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f584e-4f01-49a6-8017-fbc0a10e4fc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:56.393257 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f584e-4f01-49a6-8017-fbc0a10e4fc7 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:56.405078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664d8a4-f533-4dbc-a399-ae7a70963c11 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:56.417882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 100/192] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:37:56 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:37:56.419131 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ee9b0-0925-4dc0-8e4e-a2935aff2536 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:56.419421 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ee9b0-0925-4dc0-8e4e-a2935aff2536 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:56.419697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ee9b0-0925-4dc0-8e4e-a2935aff2536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:56.436519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-765ee9b0-0925-4dc0-8e4e-a2935aff2536 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:56.438277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-765ee9b0-0925-4dc0-8e4e-a2935aff2536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:56.442419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe90796-3191-497b-9b37-3b5669076d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:56.443347 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90796-3191-497b-9b37-3b5669076d12 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:56.443629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90796-3191-497b-9b37-3b5669076d12 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:56.443906 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90796-3191-497b-9b37-3b5669076d12 None admin] PATH_INFO: `/v3/users/fe40aadfa44049f58071a54c1101a548` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:56.459809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe90796-3191-497b-9b37-3b5669076d12 None admin] RBAC: Authorizing `identity:get_user(user_id=fe40aadfa44049f58071a54c1101a548)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:56.461339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe90796-3191-497b-9b37-3b5669076d12 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:56.467702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 93/193] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:56 2022] GET /identity/v3/users/fe40aadfa44049f58071a54c1101a548 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:56.467702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe90796-3191-497b-9b37-3b5669076d12 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:56.520104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 101/194] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:56.521288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ee9b0-0925-4dc0-8e4e-a2935aff2536 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:56.570791 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe90796-3191-497b-9b37-3b5669076d12 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:56.605619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ffae6-9a94-48f8-90a9-ed20107cea3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:56.606291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ffae6-9a94-48f8-90a9-ed20107cea3a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:56.606598 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ffae6-9a94-48f8-90a9-ed20107cea3a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:56.606868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ffae6-9a94-48f8-90a9-ed20107cea3a None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:56.607625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ffae6-9a94-48f8-90a9-ed20107cea3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:56.607931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ffae6-9a94-48f8-90a9-ed20107cea3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:56.609346 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ffae6-9a94-48f8-90a9-ed20107cea3a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:56.618510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 94/195] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:37:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:56.618510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ffae6-9a94-48f8-90a9-ed20107cea3a None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:56.637617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ee9b0-0925-4dc0-8e4e-a2935aff2536 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:56.677761 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42e66e-8137-489d-aeef-a12715987521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:56.678410 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42e66e-8137-489d-aeef-a12715987521 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:56.678690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42e66e-8137-489d-aeef-a12715987521 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:56.679064 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42e66e-8137-489d-aeef-a12715987521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:56.733895 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ffae6-9a94-48f8-90a9-ed20107cea3a None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:56.759452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42e66e-8137-489d-aeef-a12715987521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:56.761360 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42e66e-8137-489d-aeef-a12715987521 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:56.769044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394cc28-d332-4601-bbcc-0d3ff0c2b7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:56.770224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394cc28-d332-4601-bbcc-0d3ff0c2b7f7 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:56.770224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394cc28-d332-4601-bbcc-0d3ff0c2b7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:56.770390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394cc28-d332-4601-bbcc-0d3ff0c2b7f7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:56.780936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394cc28-d332-4601-bbcc-0d3ff0c2b7f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-9394cc28-d332-4601-bbcc-0d3ff0c2b7f7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:56.788637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:56.789533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:56.791358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:56.791358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:56.791358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:56.791358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:56.791358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:56.791358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:56.791358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:56.791358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:56.791358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:56.791358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:56.791358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:37:56.791358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:37:56.791358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 95/196] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:56.791358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394cc28-d332-4601-bbcc-0d3ff0c2b7f7 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:56.865900 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 102/197] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:37:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:37:56.867199 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42e66e-8137-489d-aeef-a12715987521 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:56.896071 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394cc28-d332-4601-bbcc-0d3ff0c2b7f7 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:56.929594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd378e1-61d7-49f4-86d7-01c0fa7ad597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:56.930502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd378e1-61d7-49f4-86d7-01c0fa7ad597 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:56.930790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd378e1-61d7-49f4-86d7-01c0fa7ad597 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:56.931076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd378e1-61d7-49f4-86d7-01c0fa7ad597 None admin] PATH_INFO: `/v3/users/fe40aadfa44049f58071a54c1101a548` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:56.932259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd378e1-61d7-49f4-86d7-01c0fa7ad597 None admin] RBAC: Authorizing `identity:get_user(user_id=fe40aadfa44049f58071a54c1101a548)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:56.933802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd378e1-61d7-49f4-86d7-01c0fa7ad597 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:56.935697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 96/198] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:56 2022] GET /identity/v3/users/fe40aadfa44049f58071a54c1101a548 => generated 283 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:56.936469 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd378e1-61d7-49f4-86d7-01c0fa7ad597 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:56.969943 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42e66e-8137-489d-aeef-a12715987521 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:57.005489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0a467-158a-413b-bf0f-033bbefb3929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:57.006990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0a467-158a-413b-bf0f-033bbefb3929 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:57.007476 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0a467-158a-413b-bf0f-033bbefb3929 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:57.007970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0a467-158a-413b-bf0f-033bbefb3929 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:57.008913 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0a467-158a-413b-bf0f-033bbefb3929 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:57.009418 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0a467-158a-413b-bf0f-033bbefb3929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:57.011021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0a467-158a-413b-bf0f-033bbefb3929 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:57.019746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 103/199] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:37:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:57.020985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0a467-158a-413b-bf0f-033bbefb3929 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:57.039227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd378e1-61d7-49f4-86d7-01c0fa7ad597 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:57.073403 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12389b9-badc-48ce-874f-258ebe8b0628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:57.074257 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12389b9-badc-48ce-874f-258ebe8b0628 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:57.074538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12389b9-badc-48ce-874f-258ebe8b0628 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:57.074860 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12389b9-badc-48ce-874f-258ebe8b0628 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:57.076045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12389b9-badc-48ce-874f-258ebe8b0628 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:57.077494 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12389b9-badc-48ce-874f-258ebe8b0628 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:57.079439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 97/200] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:56 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:57.079998 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12389b9-badc-48ce-874f-258ebe8b0628 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:57.124131 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0a467-158a-413b-bf0f-033bbefb3929 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:57.160002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25301dd-7bbb-41a4-a267-64f3b5f1e178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:57.160907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25301dd-7bbb-41a4-a267-64f3b5f1e178 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:57.161393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25301dd-7bbb-41a4-a267-64f3b5f1e178 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:57.161941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25301dd-7bbb-41a4-a267-64f3b5f1e178 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:57.186270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12389b9-badc-48ce-874f-258ebe8b0628 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:57.220700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66347e-2918-45ac-a599-3ed15a37fbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:57.221567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66347e-2918-45ac-a599-3ed15a37fbe6 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:57.221847 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66347e-2918-45ac-a599-3ed15a37fbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:57.222119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66347e-2918-45ac-a599-3ed15a37fbe6 None admin] PATH_INFO: `/v3/users/1984a1910cb74be89a85e92df2957b48` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:57.223343 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66347e-2918-45ac-a599-3ed15a37fbe6 None admin] RBAC: Authorizing `identity:get_user(user_id=1984a1910cb74be89a85e92df2957b48)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:57.224804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66347e-2918-45ac-a599-3ed15a37fbe6 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:57.226419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 98/201] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:57 2022] GET /identity/v3/users/1984a1910cb74be89a85e92df2957b48 => generated 289 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:57.227279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66347e-2918-45ac-a599-3ed15a37fbe6 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:57.233422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25301dd-7bbb-41a4-a267-64f3b5f1e178 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:57.235758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25301dd-7bbb-41a4-a267-64f3b5f1e178 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:57.324594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66347e-2918-45ac-a599-3ed15a37fbe6 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:57.335349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 104/202] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:37:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:37:57.336542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25301dd-7bbb-41a4-a267-64f3b5f1e178 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:57.358219 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b21d8c-f0c2-45e5-a82a-5492a612483b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:57.358912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b21d8c-f0c2-45e5-a82a-5492a612483b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:57.359253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b21d8c-f0c2-45e5-a82a-5492a612483b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:57.360037 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b21d8c-f0c2-45e5-a82a-5492a612483b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:57.360862 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b21d8c-f0c2-45e5-a82a-5492a612483b None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=a77cd5b44fbe457d82395d5a96360089, user.id=fe40aadfa44049f58071a54c1101a548) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:57.361171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b21d8c-f0c2-45e5-a82a-5492a612483b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:57.362694 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b21d8c-f0c2-45e5-a82a-5492a612483b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:57.368493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 99/203] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:37:57 2022] GET /identity/v3/role_assignments?user.id=fe40aadfa44049f58071a54c1101a548&scope.project.id=a77cd5b44fbe457d82395d5a96360089&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:57.369291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b21d8c-f0c2-45e5-a82a-5492a612483b None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:57.442100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25301dd-7bbb-41a4-a267-64f3b5f1e178 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:57.479403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef524947-5ab0-4d6d-a41f-28f057cb019a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:57.480444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef524947-5ab0-4d6d-a41f-28f057cb019a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:57.480929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef524947-5ab0-4d6d-a41f-28f057cb019a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:57.481446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef524947-5ab0-4d6d-a41f-28f057cb019a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:57.560420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef524947-5ab0-4d6d-a41f-28f057cb019a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:57.564081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef524947-5ab0-4d6d-a41f-28f057cb019a None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:57.605987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b21d8c-f0c2-45e5-a82a-5492a612483b None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:57.668639 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba801f-5308-4c56-bfbd-5e8c9301f5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:57.669849 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba801f-5308-4c56-bfbd-5e8c9301f5e8 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:57.671055 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba801f-5308-4c56-bfbd-5e8c9301f5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:57.671538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba801f-5308-4c56-bfbd-5e8c9301f5e8 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:57.677430 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba801f-5308-4c56-bfbd-5e8c9301f5e8 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:57.679158 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba801f-5308-4c56-bfbd-5e8c9301f5e8 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:57.681089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 100/204] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:57 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:57.687930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba801f-5308-4c56-bfbd-5e8c9301f5e8 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:57.733031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 105/205] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:37:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:37:57.738740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef524947-5ab0-4d6d-a41f-28f057cb019a None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:57.831127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba801f-5308-4c56-bfbd-5e8c9301f5e8 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:57.845691 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef524947-5ab0-4d6d-a41f-28f057cb019a None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:57.865788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b76d4-63d3-4df7-b13b-a1693ae176c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:57.866844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b76d4-63d3-4df7-b13b-a1693ae176c9 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:57.867332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b76d4-63d3-4df7-b13b-a1693ae176c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:57.867809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b76d4-63d3-4df7-b13b-a1693ae176c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:57.868842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b76d4-63d3-4df7-b13b-a1693ae176c9 None admin] RBAC: Adding query filter params (role.id=a29f4364896c46dd84dcee5b969b9898, scope.project.id=a77cd5b44fbe457d82395d5a96360089, user.id=fe40aadfa44049f58071a54c1101a548) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:57.869372 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b76d4-63d3-4df7-b13b-a1693ae176c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:57.871043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b76d4-63d3-4df7-b13b-a1693ae176c9 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:57.877501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 101/206] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:37:57 2022] GET /identity/v3/role_assignments?user.id=fe40aadfa44049f58071a54c1101a548&scope.project.id=a77cd5b44fbe457d82395d5a96360089&role.id=a29f4364896c46dd84dcee5b969b9898 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:57.892687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d49b3-ba37-41e5-9fe9-93fba2a02486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:57.893580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d49b3-ba37-41e5-9fe9-93fba2a02486 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:57.893859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d49b3-ba37-41e5-9fe9-93fba2a02486 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:57.894132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d49b3-ba37-41e5-9fe9-93fba2a02486 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:57.895372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d49b3-ba37-41e5-9fe9-93fba2a02486 None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:57.896828 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d49b3-ba37-41e5-9fe9-93fba2a02486 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:57.898930 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 106/207] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:57 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:57.904585 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b76d4-63d3-4df7-b13b-a1693ae176c9 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:58.059530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b76d4-63d3-4df7-b13b-a1693ae176c9 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:58.102341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3928c2c-c9f1-47e7-8df4-6f64e1a708fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:58.103395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3928c2c-c9f1-47e7-8df4-6f64e1a708fd None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:58.103880 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3928c2c-c9f1-47e7-8df4-6f64e1a708fd None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:58.104374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3928c2c-c9f1-47e7-8df4-6f64e1a708fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:58.105404 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3928c2c-c9f1-47e7-8df4-6f64e1a708fd None admin] RBAC: Adding query filter params (role.id=94938baf3b7443bd8e53ce11902c7247, scope.project.id=80047f4aaba64866971a8b97d21dee0a, user.id=1984a1910cb74be89a85e92df2957b48) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:58.105916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3928c2c-c9f1-47e7-8df4-6f64e1a708fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:58.107573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3928c2c-c9f1-47e7-8df4-6f64e1a708fd None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:58.114144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 102/208] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:37:57 2022] GET /identity/v3/role_assignments?user.id=1984a1910cb74be89a85e92df2957b48&scope.project.id=80047f4aaba64866971a8b97d21dee0a&role.id=94938baf3b7443bd8e53ce11902c7247 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:58.934853 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a6427-65c1-4747-9930-9cb0981ba11d None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:58.935377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a6427-65c1-4747-9930-9cb0981ba11d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:58.935377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a6427-65c1-4747-9930-9cb0981ba11d None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:58.936996 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 107/209] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:58.942981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f708-a848-441a-bd21-4ade44e5fec7 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:58.943802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f708-a848-441a-bd21-4ade44e5fec7 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:58.944272 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f708-a848-441a-bd21-4ade44e5fec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:58.961727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-4a19f708-a848-441a-bd21-4ade44e5fec7 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:58.963191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-4a19f708-a848-441a-bd21-4ade44e5fec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:59.027988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 103/210] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:37:59.053850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a6427-65c1-4747-9930-9cb0981ba11d None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:59.095113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fda3e-1f01-4f49-b267-7dcbe4919600 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:59.095401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fda3e-1f01-4f49-b267-7dcbe4919600 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:59.095915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fda3e-1f01-4f49-b267-7dcbe4919600 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:59.098924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 104/211] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:37:59.102965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be5c75-101d-4834-b8b3-9221b2783295 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:59.103460 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be5c75-101d-4834-b8b3-9221b2783295 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:59.103932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be5c75-101d-4834-b8b3-9221b2783295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:59.121892 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e2be5c75-101d-4834-b8b3-9221b2783295 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:59.123391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e2be5c75-101d-4834-b8b3-9221b2783295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:59.157143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a6427-65c1-4747-9930-9cb0981ba11d None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:59.191964 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0916d2f-b400-4032-827b-21500f2e2fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:59.192652 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 105/212] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:37:59.192987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0916d2f-b400-4032-827b-21500f2e2fb6 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:59.193267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0916d2f-b400-4032-827b-21500f2e2fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:59.193540 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0916d2f-b400-4032-827b-21500f2e2fb6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:59.204132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0916d2f-b400-4032-827b-21500f2e2fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-e0916d2f-b400-4032-827b-21500f2e2fb6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:37:59.211938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:37:59.212865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:37:59.213631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:37:59.213631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:37:59.213631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:37:59.213631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:37:59.213631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:37:59.213631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:37:59.213631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:37:59.213631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:37:59.213631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:37:59.213631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:37:59.213631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:37:59.213631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:37:59.216013 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 108/213] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:37:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:37:59.221600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be5c75-101d-4834-b8b3-9221b2783295 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:59.223749 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0916d2f-b400-4032-827b-21500f2e2fb6 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:59.330282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be5c75-101d-4834-b8b3-9221b2783295 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:59.347858 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0916d2f-b400-4032-827b-21500f2e2fb6 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:59.364294 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4005f3-82c0-43c9-87ee-901d4c24b1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:59.365373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4005f3-82c0-43c9-87ee-901d4c24b1e6 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:59.365833 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4005f3-82c0-43c9-87ee-901d4c24b1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:59.366281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4005f3-82c0-43c9-87ee-901d4c24b1e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:59.367646 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4005f3-82c0-43c9-87ee-901d4c24b1e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:59.369248 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4005f3-82c0-43c9-87ee-901d4c24b1e6 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:59.375333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 106/214] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:37:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:59.377411 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4005f3-82c0-43c9-87ee-901d4c24b1e6 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:59.398210 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d260644-c4d3-444e-8029-d91bdaafd30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:59.398937 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d260644-c4d3-444e-8029-d91bdaafd30f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:59.399216 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d260644-c4d3-444e-8029-d91bdaafd30f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:59.399490 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d260644-c4d3-444e-8029-d91bdaafd30f None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:59.400239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d260644-c4d3-444e-8029-d91bdaafd30f None admin] RBAC: Adding query filter params (name=member) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:37:59.400542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d260644-c4d3-444e-8029-d91bdaafd30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:59.402333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d260644-c4d3-444e-8029-d91bdaafd30f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:59.413444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 109/215] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:37:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:37:59.418831 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d260644-c4d3-444e-8029-d91bdaafd30f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:59.497948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4005f3-82c0-43c9-87ee-901d4c24b1e6 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:59.537987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d260644-c4d3-444e-8029-d91bdaafd30f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:59.540185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55051b0-e53d-42c0-beab-d6fefd30f34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:59.541089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55051b0-e53d-42c0-beab-d6fefd30f34c None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:59.541567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55051b0-e53d-42c0-beab-d6fefd30f34c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:59.542066 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55051b0-e53d-42c0-beab-d6fefd30f34c None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:59.543021 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55051b0-e53d-42c0-beab-d6fefd30f34c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:59.546914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55051b0-e53d-42c0-beab-d6fefd30f34c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:59.572167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 107/216] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:37:59 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:37:59.593247 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c8d92-2e4c-4c5d-ab93-bd6493feeb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:59.594148 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c8d92-2e4c-4c5d-ab93-bd6493feeb37 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:59.594476 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c8d92-2e4c-4c5d-ab93-bd6493feeb37 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:59.594776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c8d92-2e4c-4c5d-ab93-bd6493feeb37 None admin] PATH_INFO: `/v3/users/723f9af578c44d16a23c8dd395d19b38` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:59.610318 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c8d92-2e4c-4c5d-ab93-bd6493feeb37 None admin] RBAC: Authorizing `identity:get_user(user_id=723f9af578c44d16a23c8dd395d19b38)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:59.611876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c8d92-2e4c-4c5d-ab93-bd6493feeb37 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:59.613746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 110/217] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:37:59 2022] GET /identity/v3/users/723f9af578c44d16a23c8dd395d19b38 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:37:59.619554 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55051b0-e53d-42c0-beab-d6fefd30f34c None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:59.733248 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55051b0-e53d-42c0-beab-d6fefd30f34c None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:37:59.770592 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21a4d1-4751-4796-85a3-87dabe667ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:37:59.771236 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21a4d1-4751-4796-85a3-87dabe667ea3 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:59.771520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21a4d1-4751-4796-85a3-87dabe667ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:59.771795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21a4d1-4751-4796-85a3-87dabe667ea3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:59.841378 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21a4d1-4751-4796-85a3-87dabe667ea3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:37:59.842889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21a4d1-4751-4796-85a3-87dabe667ea3 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:37:59.856345 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362fc3e-faf7-481d-a48f-1b9a3c958038 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:59.856631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362fc3e-faf7-481d-a48f-1b9a3c958038 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:59.856913 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362fc3e-faf7-481d-a48f-1b9a3c958038 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:59.858289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 111/218] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:37:59.864646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190397f-d01d-4bf8-ab6c-c206e150c738 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:37:59.864929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190397f-d01d-4bf8-ab6c-c206e150c738 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:37:59.865205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190397f-d01d-4bf8-ab6c-c206e150c738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:37:59.890734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-9190397f-d01d-4bf8-ab6c-c206e150c738 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:37:59.891936 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-9190397f-d01d-4bf8-ab6c-c206e150c738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:37:59.952635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 108/219] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:37:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:37:59.958250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21a4d1-4751-4796-85a3-87dabe667ea3 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:37:59.976902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 112/220] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:00.004835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190397f-d01d-4bf8-ab6c-c206e150c738 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:00.065440 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21a4d1-4751-4796-85a3-87dabe667ea3 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:00.100191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d271a8-569e-4291-b88b-e1e7e994fc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:00.101061 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d271a8-569e-4291-b88b-e1e7e994fc54 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:00.101342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d271a8-569e-4291-b88b-e1e7e994fc54 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:00.101621 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d271a8-569e-4291-b88b-e1e7e994fc54 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:00.102857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d271a8-569e-4291-b88b-e1e7e994fc54 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:00.104335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d271a8-569e-4291-b88b-e1e7e994fc54 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:00.106063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 109/221] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:37:59 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:00.112690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d271a8-569e-4291-b88b-e1e7e994fc54 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:00.124706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190397f-d01d-4bf8-ab6c-c206e150c738 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:00.194282 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924310e-0080-4bcc-9246-fd01f2ce48c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:00.195112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924310e-0080-4bcc-9246-fd01f2ce48c3 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:00.195393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924310e-0080-4bcc-9246-fd01f2ce48c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:00.195667 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924310e-0080-4bcc-9246-fd01f2ce48c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:00.212717 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d271a8-569e-4291-b88b-e1e7e994fc54 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:00.223657 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924310e-0080-4bcc-9246-fd01f2ce48c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-f924310e-0080-4bcc-9246-fd01f2ce48c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:00.231348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:00.232224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:00.232992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:00.232992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:00.232992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:00.232992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:00.232992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:00.232992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:00.232992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:00.232992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:00.232992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:00.232992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:00.232992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:38:00.232992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:00.236402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 113/222] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:38:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:00.241650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924310e-0080-4bcc-9246-fd01f2ce48c3 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:00.252191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d6f4e-4350-4a7f-ac5f-caa987df659f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:00.252851 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d6f4e-4350-4a7f-ac5f-caa987df659f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:00.253396 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d6f4e-4350-4a7f-ac5f-caa987df659f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:00.253675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d6f4e-4350-4a7f-ac5f-caa987df659f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:00.254510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d6f4e-4350-4a7f-ac5f-caa987df659f None admin] RBAC: Adding query filter params (role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d, scope.project.id=a77cd5b44fbe457d82395d5a96360089, user.id=723f9af578c44d16a23c8dd395d19b38) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:00.259648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d6f4e-4350-4a7f-ac5f-caa987df659f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:00.261147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d6f4e-4350-4a7f-ac5f-caa987df659f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:00.270230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 110/223] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:00 2022] GET /identity/v3/role_assignments?user.id=723f9af578c44d16a23c8dd395d19b38&scope.project.id=a77cd5b44fbe457d82395d5a96360089&role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:00.417224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924310e-0080-4bcc-9246-fd01f2ce48c3 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:00.465826 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43451b5b-b0a8-42df-83ff-65d8a5e4bac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:00.467061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43451b5b-b0a8-42df-83ff-65d8a5e4bac5 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:00.467545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43451b5b-b0a8-42df-83ff-65d8a5e4bac5 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:00.468035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43451b5b-b0a8-42df-83ff-65d8a5e4bac5 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:00.469192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43451b5b-b0a8-42df-83ff-65d8a5e4bac5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:00.469723 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43451b5b-b0a8-42df-83ff-65d8a5e4bac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:00.471460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43451b5b-b0a8-42df-83ff-65d8a5e4bac5 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:00.487301 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 114/224] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:00.492699 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d6f4e-4350-4a7f-ac5f-caa987df659f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:00.591762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3b37b-844d-49e8-a9eb-3daf974aac8e None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:00.592502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3b37b-844d-49e8-a9eb-3daf974aac8e None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:00.592987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3b37b-844d-49e8-a9eb-3daf974aac8e None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:00.594598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 115/225] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:00.600062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d48ec3-195e-4526-8cc4-1a1a3836d398 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:00.600574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d48ec3-195e-4526-8cc4-1a1a3836d398 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:00.601067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d48ec3-195e-4526-8cc4-1a1a3836d398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:00.620987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-50d48ec3-195e-4526-8cc4-1a1a3836d398 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:00.623016 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-50d48ec3-195e-4526-8cc4-1a1a3836d398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:00.625877 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d6f4e-4350-4a7f-ac5f-caa987df659f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:00.683631 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a579b1-adaf-437c-9f14-0d79b38f3a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:00.684500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a579b1-adaf-437c-9f14-0d79b38f3a5d None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:00.684777 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a579b1-adaf-437c-9f14-0d79b38f3a5d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:00.685042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a579b1-adaf-437c-9f14-0d79b38f3a5d None admin] PATH_INFO: `/v3/users/fe40aadfa44049f58071a54c1101a548` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:00.688793 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a579b1-adaf-437c-9f14-0d79b38f3a5d None admin] RBAC: Authorizing `identity:get_user(user_id=fe40aadfa44049f58071a54c1101a548)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:00.690261 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a579b1-adaf-437c-9f14-0d79b38f3a5d None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:00.692112 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 111/226] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:00 2022] GET /identity/v3/users/fe40aadfa44049f58071a54c1101a548 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:00.699699 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a579b1-adaf-437c-9f14-0d79b38f3a5d None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:00.746427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 116/227] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:00.772730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d48ec3-195e-4526-8cc4-1a1a3836d398 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:00.880517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a579b1-adaf-437c-9f14-0d79b38f3a5d None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:00.910395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d48ec3-195e-4526-8cc4-1a1a3836d398 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:00.915205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e5f4d-0d18-43e7-95d3-2054c75a74e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:00.916064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5f4d-0d18-43e7-95d3-2054c75a74e7 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:00.916344 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5f4d-0d18-43e7-95d3-2054c75a74e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:00.916618 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5f4d-0d18-43e7-95d3-2054c75a74e7 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:00.917810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e5f4d-0d18-43e7-95d3-2054c75a74e7 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:00.922748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e5f4d-0d18-43e7-95d3-2054c75a74e7 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:00.922748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 112/228] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:00 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:00.927526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e5f4d-0d18-43e7-95d3-2054c75a74e7 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:00.987524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f0470-897e-4c9c-8440-9a352ce55472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:00.988294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f0470-897e-4c9c-8440-9a352ce55472 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:00.988732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f0470-897e-4c9c-8440-9a352ce55472 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:00.989012 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f0470-897e-4c9c-8440-9a352ce55472 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:01.008902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f0470-897e-4c9c-8440-9a352ce55472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-696f0470-897e-4c9c-8440-9a352ce55472 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:01.021022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:01.021933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:01.022756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:01.022756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:01.022756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:01.022756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:01.022756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:01.022756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:01.022756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:01.022756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:01.022756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:01.022756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:01.022756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:01.022756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:01.025537 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 117/229] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:38:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:01.030658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f0470-897e-4c9c-8440-9a352ce55472 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:01.048211 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e5f4d-0d18-43e7-95d3-2054c75a74e7 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:01.081985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae90a58-71da-403b-8396-f285c0e789ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:01.083270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae90a58-71da-403b-8396-f285c0e789ee None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:01.083778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae90a58-71da-403b-8396-f285c0e789ee None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:01.084255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae90a58-71da-403b-8396-f285c0e789ee None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089/users/fe40aadfa44049f58071a54c1101a548/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:01.086552 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae90a58-71da-403b-8396-f285c0e789ee None admin] RBAC: Authorizing `identity:create_grant(project_id=a77cd5b44fbe457d82395d5a96360089, user_id=fe40aadfa44049f58071a54c1101a548, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:01.088238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae90a58-71da-403b-8396-f285c0e789ee None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:01.110519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 113/230] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:38:00 2022] PUT /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089/users/fe40aadfa44049f58071a54c1101a548/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:38:01.205166 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f0470-897e-4c9c-8440-9a352ce55472 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:01.250966 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4828b-0c76-43de-a24a-965127061eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:01.251570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4828b-0c76-43de-a24a-965127061eb5 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:01.251852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4828b-0c76-43de-a24a-965127061eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:01.252123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4828b-0c76-43de-a24a-965127061eb5 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:01.252880 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4828b-0c76-43de-a24a-965127061eb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:01.253455 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4828b-0c76-43de-a24a-965127061eb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:01.254902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4828b-0c76-43de-a24a-965127061eb5 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:01.269846 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 118/231] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:01.275790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae90a58-71da-403b-8396-f285c0e789ee None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:01.383986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae90a58-71da-403b-8396-f285c0e789ee None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:01.420876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88438c-9717-4889-ae96-46eda6ccc46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:01.421834 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88438c-9717-4889-ae96-46eda6ccc46e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:01.422113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88438c-9717-4889-ae96-46eda6ccc46e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:01.422382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88438c-9717-4889-ae96-46eda6ccc46e None admin] PATH_INFO: `/v3/users/fe40aadfa44049f58071a54c1101a548` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:01.423632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88438c-9717-4889-ae96-46eda6ccc46e None admin] RBAC: Authorizing `identity:get_user(user_id=fe40aadfa44049f58071a54c1101a548)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:01.425075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88438c-9717-4889-ae96-46eda6ccc46e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:01.426900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 114/232] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:01 2022] GET /identity/v3/users/fe40aadfa44049f58071a54c1101a548 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:01.432286 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4828b-0c76-43de-a24a-965127061eb5 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:01.536727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4828b-0c76-43de-a24a-965127061eb5 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:01.572281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca185acb-fa39-4b64-8f12-12fda8f8ab27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:01.573315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca185acb-fa39-4b64-8f12-12fda8f8ab27 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:01.573601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca185acb-fa39-4b64-8f12-12fda8f8ab27 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:01.573875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca185acb-fa39-4b64-8f12-12fda8f8ab27 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:01.575089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca185acb-fa39-4b64-8f12-12fda8f8ab27 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:01.576533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca185acb-fa39-4b64-8f12-12fda8f8ab27 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:01.578808 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 119/233] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:01 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:01.584555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88438c-9717-4889-ae96-46eda6ccc46e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:01.706960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88438c-9717-4889-ae96-46eda6ccc46e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:01.755264 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d90d80-85f2-417d-9a5a-86e1c5315842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:01.756159 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d90d80-85f2-417d-9a5a-86e1c5315842 None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:01.756438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d90d80-85f2-417d-9a5a-86e1c5315842 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:01.756711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d90d80-85f2-417d-9a5a-86e1c5315842 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089/users/fe40aadfa44049f58071a54c1101a548/roles/a29f4364896c46dd84dcee5b969b9898` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:01.758993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d90d80-85f2-417d-9a5a-86e1c5315842 None admin] RBAC: Authorizing `identity:create_grant(project_id=a77cd5b44fbe457d82395d5a96360089, user_id=fe40aadfa44049f58071a54c1101a548, role_id=a29f4364896c46dd84dcee5b969b9898)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:01.760459 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d90d80-85f2-417d-9a5a-86e1c5315842 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:01.777144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 115/234] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:38:01 2022] PUT /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089/users/fe40aadfa44049f58071a54c1101a548/roles/a29f4364896c46dd84dcee5b969b9898 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:38:02.552299 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188d654-fe23-4ca3-93ca-6da440d76169 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:02.552299 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188d654-fe23-4ca3-93ca-6da440d76169 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:02.552299 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188d654-fe23-4ca3-93ca-6da440d76169 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:02.553210 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 120/235] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:02.558649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea446c29-c711-4b01-a843-b785661191b6 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:02.559418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea446c29-c711-4b01-a843-b785661191b6 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:02.559837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea446c29-c711-4b01-a843-b785661191b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:02.575343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4828e6-90a8-4944-b2ca-d55225765493 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:02.575843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4828e6-90a8-4944-b2ca-d55225765493 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:02.576260 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4828e6-90a8-4944-b2ca-d55225765493 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:02.578393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 121/236] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:02.583432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608d43d-d97a-42df-9693-a8fba4215049 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:02.583713 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608d43d-d97a-42df-9693-a8fba4215049 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:02.584186 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608d43d-d97a-42df-9693-a8fba4215049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:02.589240 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-ea446c29-c711-4b01-a843-b785661191b6 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:02.590416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ea446c29-c711-4b01-a843-b785661191b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:02.610689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-a608d43d-d97a-42df-9693-a8fba4215049 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:02.611895 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a608d43d-d97a-42df-9693-a8fba4215049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:02.698940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 116/237] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:02.722114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea446c29-c711-4b01-a843-b785661191b6 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:02.725016 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 122/238] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:02.750376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608d43d-d97a-42df-9693-a8fba4215049 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:02.906159 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea446c29-c711-4b01-a843-b785661191b6 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:02.931350 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608d43d-d97a-42df-9693-a8fba4215049 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:02.960812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a89f2d-aca1-47bd-94c3-32e0502a795f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:02.961615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a89f2d-aca1-47bd-94c3-32e0502a795f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:02.961899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a89f2d-aca1-47bd-94c3-32e0502a795f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:02.962174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a89f2d-aca1-47bd-94c3-32e0502a795f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:02.972841 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a89f2d-aca1-47bd-94c3-32e0502a795f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:02.973023 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea7fac-68ea-4292-a650-a207b56bdf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:02.973821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7fac-68ea-4292-a650-a207b56bdf85 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:02.974100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7fac-68ea-4292-a650-a207b56bdf85 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:02.974375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7fac-68ea-4292-a650-a207b56bdf85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:02.975584 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea7fac-68ea-4292-a650-a207b56bdf85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:02.977021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea7fac-68ea-4292-a650-a207b56bdf85 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:02.978943 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 123/239] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:02.980356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078894e-404e-4529-b121-441e2a38c06f None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:02.980636 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078894e-404e-4529-b121-441e2a38c06f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:02.980909 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078894e-404e-4529-b121-441e2a38c06f None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:02.982117 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 124/240] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:02.983702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078894e-404e-4529-b121-441e2a38c06f None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-98a89f2d-aca1-47bd-94c3-32e0502a795f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:02.985201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:02.986051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:02.986933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:02.986933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:02.986933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:02.986933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:02.986933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:02.986933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:02.986933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:02.986933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:02.986933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:02.986933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:02.986933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:38:02.986933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:02.990030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 117/241] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:02.992636 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5e67b-7195-4e5e-a646-03349ab7ad6f None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:02.993141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5e67b-7195-4e5e-a646-03349ab7ad6f None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:02.993630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5e67b-7195-4e5e-a646-03349ab7ad6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:03.014598 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-1cc5e67b-7195-4e5e-a646-03349ab7ad6f None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:03.016635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-1cc5e67b-7195-4e5e-a646-03349ab7ad6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:03.089410 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078894e-404e-4529-b121-441e2a38c06f None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:03.098146 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 118/242] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:03.099257 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5e67b-7195-4e5e-a646-03349ab7ad6f None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:03.145307 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1c6d6-fdee-4a1e-84bb-a52a16169726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:03.146652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1c6d6-fdee-4a1e-84bb-a52a16169726 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:03.147363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1c6d6-fdee-4a1e-84bb-a52a16169726 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:03.148017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1c6d6-fdee-4a1e-84bb-a52a16169726 None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:03.149367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1c6d6-fdee-4a1e-84bb-a52a16169726 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:03.152136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1c6d6-fdee-4a1e-84bb-a52a16169726 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:03.182435 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 125/243] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:38:02 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:38:03.183731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1c6d6-fdee-4a1e-84bb-a52a16169726 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:03.208661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5e67b-7195-4e5e-a646-03349ab7ad6f None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:03.245042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31877f71-870c-4f45-a712-f53c2b72d128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:03.245741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31877f71-870c-4f45-a712-f53c2b72d128 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:03.246009 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31877f71-870c-4f45-a712-f53c2b72d128 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:03.246288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31877f71-870c-4f45-a712-f53c2b72d128 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:03.247089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31877f71-870c-4f45-a712-f53c2b72d128 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:03.247615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31877f71-870c-4f45-a712-f53c2b72d128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:03.249259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31877f71-870c-4f45-a712-f53c2b72d128 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:03.259077 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 119/244] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:03.269564 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31877f71-870c-4f45-a712-f53c2b72d128 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:03.318877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1c6d6-fdee-4a1e-84bb-a52a16169726 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:03.359668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4ad2f-a14a-4427-8b23-ad0cb1ad4d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:03.360479 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4ad2f-a14a-4427-8b23-ad0cb1ad4d2e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:03.360762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4ad2f-a14a-4427-8b23-ad0cb1ad4d2e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:03.362552 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4ad2f-a14a-4427-8b23-ad0cb1ad4d2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:03.384530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4ad2f-a14a-4427-8b23-ad0cb1ad4d2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-59c4ad2f-a14a-4427-8b23-ad0cb1ad4d2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:03.393070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:03.394002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:03.394803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:03.394803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:03.394803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:03.394803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:03.394803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:03.394803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:03.394803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:03.394803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:03.394803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:03.394803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:03.394803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:03.394803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:03.394803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 126/245] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:03.406052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4ad2f-a14a-4427-8b23-ad0cb1ad4d2e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:03.442268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31877f71-870c-4f45-a712-f53c2b72d128 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:03.478894 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9006532-311d-4f40-8e76-4eda122754ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:03.480044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9006532-311d-4f40-8e76-4eda122754ef None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:03.480579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9006532-311d-4f40-8e76-4eda122754ef None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:03.481009 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9006532-311d-4f40-8e76-4eda122754ef None admin] PATH_INFO: `/v3/users/4894956f37c14da1ab943d20a4edcc4a` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:03.496807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9006532-311d-4f40-8e76-4eda122754ef None admin] RBAC: Authorizing `identity:get_user(user_id=4894956f37c14da1ab943d20a4edcc4a)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:03.498502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9006532-311d-4f40-8e76-4eda122754ef None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:03.501134 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 120/246] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:03 2022] GET /identity/v3/users/4894956f37c14da1ab943d20a4edcc4a => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:03.506452 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9006532-311d-4f40-8e76-4eda122754ef None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:03.561130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4ad2f-a14a-4427-8b23-ad0cb1ad4d2e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:03.616663 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b0c51-d208-4550-a7fa-a9ed40f22308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:03.617602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0c51-d208-4550-a7fa-a9ed40f22308 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:03.617888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0c51-d208-4550-a7fa-a9ed40f22308 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:03.618163 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0c51-d208-4550-a7fa-a9ed40f22308 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:03.618952 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b0c51-d208-4550-a7fa-a9ed40f22308 None admin] RBAC: Adding query filter params (name=member) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:03.619260 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b0c51-d208-4550-a7fa-a9ed40f22308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:03.620658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b0c51-d208-4550-a7fa-a9ed40f22308 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:03.624565 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9006532-311d-4f40-8e76-4eda122754ef None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:03.632801 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 127/247] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:03.638289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b0c51-d208-4550-a7fa-a9ed40f22308 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:03.669410 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19b25-83d1-4ccb-8d51-1e3e263cc155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:03.670529 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19b25-83d1-4ccb-8d51-1e3e263cc155 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:03.670971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19b25-83d1-4ccb-8d51-1e3e263cc155 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:03.671382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19b25-83d1-4ccb-8d51-1e3e263cc155 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:03.759845 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b0c51-d208-4550-a7fa-a9ed40f22308 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:03.763194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e19b25-83d1-4ccb-8d51-1e3e263cc155 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:03.764689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e19b25-83d1-4ccb-8d51-1e3e263cc155 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:03.804556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8944334-eb77-4822-ae9f-63da0e63c66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:03.805773 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8944334-eb77-4822-ae9f-63da0e63c66e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:03.806250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8944334-eb77-4822-ae9f-63da0e63c66e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:03.806650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8944334-eb77-4822-ae9f-63da0e63c66e None admin] PATH_INFO: `/v3/users/723f9af578c44d16a23c8dd395d19b38` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:03.808438 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8944334-eb77-4822-ae9f-63da0e63c66e None admin] RBAC: Authorizing `identity:get_user(user_id=723f9af578c44d16a23c8dd395d19b38)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:03.811447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8944334-eb77-4822-ae9f-63da0e63c66e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:03.815401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 128/248] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:03 2022] GET /identity/v3/users/723f9af578c44d16a23c8dd395d19b38 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:03.822836 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8944334-eb77-4822-ae9f-63da0e63c66e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:03.910811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 121/249] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 21:38:03.917081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19b25-83d1-4ccb-8d51-1e3e263cc155 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:03.940800 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8944334-eb77-4822-ae9f-63da0e63c66e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:03.982448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbabc3-44f8-4f71-b792-dd228ae6c147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:03.983398 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbabc3-44f8-4f71-b792-dd228ae6c147 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:03.983727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbabc3-44f8-4f71-b792-dd228ae6c147 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:03.984047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbabc3-44f8-4f71-b792-dd228ae6c147 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:03.985281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbabc3-44f8-4f71-b792-dd228ae6c147 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:03.986806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbabc3-44f8-4f71-b792-dd228ae6c147 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:03.988546 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 129/250] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:03 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:03.996242 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbabc3-44f8-4f71-b792-dd228ae6c147 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:04.046748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19b25-83d1-4ccb-8d51-1e3e263cc155 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:04.094103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd34882-0312-418e-a266-9630bc86a13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:04.095276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd34882-0312-418e-a266-9630bc86a13f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:04.095564 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd34882-0312-418e-a266-9630bc86a13f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:04.095843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd34882-0312-418e-a266-9630bc86a13f None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:04.097207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd34882-0312-418e-a266-9630bc86a13f None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:04.099255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd34882-0312-418e-a266-9630bc86a13f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:04.101872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 122/251] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:03 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:04.103175 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da0fc1-18b5-4180-b770-9bed115aaade None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:04.103461 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da0fc1-18b5-4180-b770-9bed115aaade None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:04.103740 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da0fc1-18b5-4180-b770-9bed115aaade None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:04.105147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 123/252] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:04.106878 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da0fc1-18b5-4180-b770-9bed115aaade None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:04.124466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbabc3-44f8-4f71-b792-dd228ae6c147 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:04.173699 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b61c1-2f2a-4cf5-9ad1-958d3a58d82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:04.175980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b61c1-2f2a-4cf5-9ad1-958d3a58d82b None admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:04.176269 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b61c1-2f2a-4cf5-9ad1-958d3a58d82b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:04.176550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b61c1-2f2a-4cf5-9ad1-958d3a58d82b None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089/users/723f9af578c44d16a23c8dd395d19b38/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:04.180353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b61c1-2f2a-4cf5-9ad1-958d3a58d82b None admin] RBAC: Authorizing `identity:create_grant(project_id=a77cd5b44fbe457d82395d5a96360089, user_id=723f9af578c44d16a23c8dd395d19b38, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:04.183609 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b61c1-2f2a-4cf5-9ad1-958d3a58d82b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:04.211935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 130/253] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:38:03 2022] PUT /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089/users/723f9af578c44d16a23c8dd395d19b38/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:38:04.213271 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf28e73-3a26-4643-976b-93760ce8e04e None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:04.213568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf28e73-3a26-4643-976b-93760ce8e04e None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:04.214033 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf28e73-3a26-4643-976b-93760ce8e04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:04.246975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-8bf28e73-3a26-4643-976b-93760ce8e04e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:04.249265 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8bf28e73-3a26-4643-976b-93760ce8e04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:04.267445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da0fc1-18b5-4180-b770-9bed115aaade None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:04.334752 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a150f-f620-45a5-ba1e-0f91782a5041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:04.335383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a150f-f620-45a5-ba1e-0f91782a5041 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:04.335662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a150f-f620-45a5-ba1e-0f91782a5041 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:04.337153 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a150f-f620-45a5-ba1e-0f91782a5041 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:04.338001 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a150f-f620-45a5-ba1e-0f91782a5041 None admin] RBAC: Adding query filter params (role.id=94938baf3b7443bd8e53ce11902c7247, scope.project.id=a77cd5b44fbe457d82395d5a96360089, user.id=4894956f37c14da1ab943d20a4edcc4a) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:04.338314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a150f-f620-45a5-ba1e-0f91782a5041 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:04.340835 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a150f-f620-45a5-ba1e-0f91782a5041 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:04.355116 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 124/254] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:04 2022] GET /identity/v3/role_assignments?user.id=4894956f37c14da1ab943d20a4edcc4a&scope.project.id=a77cd5b44fbe457d82395d5a96360089&role.id=94938baf3b7443bd8e53ce11902c7247 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:04.369843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 131/255] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:04.394779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a150f-f620-45a5-ba1e-0f91782a5041 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:04.521324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a150f-f620-45a5-ba1e-0f91782a5041 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:04.565987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1f82f-29f3-4ec9-b955-45ee2c7e33ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:04.566880 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1f82f-29f3-4ec9-b955-45ee2c7e33ec None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:04.567164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1f82f-29f3-4ec9-b955-45ee2c7e33ec None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:04.567440 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1f82f-29f3-4ec9-b955-45ee2c7e33ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:04.580936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1f82f-29f3-4ec9-b955-45ee2c7e33ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-30a1f82f-29f3-4ec9-b955-45ee2c7e33ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:04.589423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:04.590438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:04.591283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:04.591283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:04.591283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:04.591283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:04.591283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:04.591283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:04.591283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:04.591283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:04.591283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:04.591283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:04.591283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:38:04.591283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:04.591283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 125/256] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:38:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:04.598606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf28e73-3a26-4643-976b-93760ce8e04e None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:04.712364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf28e73-3a26-4643-976b-93760ce8e04e None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:04.721651 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc7952-0060-44da-b794-8b9dc04894f5 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:04.722169 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc7952-0060-44da-b794-8b9dc04894f5 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:04.722255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc7952-0060-44da-b794-8b9dc04894f5 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:04.725120 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 126/257] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:04.730347 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a025b00-409e-45e9-9750-2b518e565968 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:04.730640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a025b00-409e-45e9-9750-2b518e565968 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:04.730968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a025b00-409e-45e9-9750-2b518e565968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:04.762202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-6a025b00-409e-45e9-9750-2b518e565968 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:04.764602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-6a025b00-409e-45e9-9750-2b518e565968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:04.768099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ca3e3-b62c-46d9-9834-6fe9b80ac87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:04.769862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ca3e3-b62c-46d9-9834-6fe9b80ac87e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:04.770365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ca3e3-b62c-46d9-9834-6fe9b80ac87e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:04.770876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ca3e3-b62c-46d9-9834-6fe9b80ac87e None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:04.771841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ca3e3-b62c-46d9-9834-6fe9b80ac87e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:04.774991 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ca3e3-b62c-46d9-9834-6fe9b80ac87e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:04.776623 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ca3e3-b62c-46d9-9834-6fe9b80ac87e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:04.791062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 132/258] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:04.796370 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ca3e3-b62c-46d9-9834-6fe9b80ac87e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:04.902404 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 127/259] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:04.936211 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a025b00-409e-45e9-9750-2b518e565968 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:04.995876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ca3e3-b62c-46d9-9834-6fe9b80ac87e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:05.072300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8417cc-3bbd-4e37-9850-b7c6de840edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:05.073175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8417cc-3bbd-4e37-9850-b7c6de840edd None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:05.073448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8417cc-3bbd-4e37-9850-b7c6de840edd None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:05.073718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8417cc-3bbd-4e37-9850-b7c6de840edd None admin] PATH_INFO: `/v3/users/fe40aadfa44049f58071a54c1101a548` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:05.076148 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8417cc-3bbd-4e37-9850-b7c6de840edd None admin] RBAC: Authorizing `identity:get_user(user_id=fe40aadfa44049f58071a54c1101a548)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:05.077601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8417cc-3bbd-4e37-9850-b7c6de840edd None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:05.080247 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 133/260] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:04 2022] GET /identity/v3/users/fe40aadfa44049f58071a54c1101a548 => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:05.085870 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8417cc-3bbd-4e37-9850-b7c6de840edd None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:05.146339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a025b00-409e-45e9-9750-2b518e565968 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:05.222694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8417cc-3bbd-4e37-9850-b7c6de840edd None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:05.232295 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d2c81-300a-4d6f-9cb2-ad61cfc7d7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:05.233387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d2c81-300a-4d6f-9cb2-ad61cfc7d7bb None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:05.233884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d2c81-300a-4d6f-9cb2-ad61cfc7d7bb None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:05.234920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d2c81-300a-4d6f-9cb2-ad61cfc7d7bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:05.250821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d2c81-300a-4d6f-9cb2-ad61cfc7d7bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-2d4d2c81-300a-4d6f-9cb2-ad61cfc7d7bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:05.259615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:05.260493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:05.261284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:05.261284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:05.261284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:05.261284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:05.261284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:05.261284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:05.261284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:05.261284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:05.261284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:05.261284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:05.261284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:05.261284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:05.263967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebef77-a725-4870-be62-bfab1fa27131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:05.264420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 128/261] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:38:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:05.264586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebef77-a725-4870-be62-bfab1fa27131 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:05.269319 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebef77-a725-4870-be62-bfab1fa27131 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:05.269319 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebef77-a725-4870-be62-bfab1fa27131 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:05.270488 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d2c81-300a-4d6f-9cb2-ad61cfc7d7bb None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:05.365232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebef77-a725-4870-be62-bfab1fa27131 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:05.366495 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebef77-a725-4870-be62-bfab1fa27131 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:05.402240 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d2c81-300a-4d6f-9cb2-ad61cfc7d7bb None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:05.456094 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1db3b4-1a8f-47d9-ab6a-9cb85226cfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:05.457129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1db3b4-1a8f-47d9-ab6a-9cb85226cfb0 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:05.457612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1db3b4-1a8f-47d9-ab6a-9cb85226cfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:05.458084 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1db3b4-1a8f-47d9-ab6a-9cb85226cfb0 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:05.459095 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1db3b4-1a8f-47d9-ab6a-9cb85226cfb0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:05.459639 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1db3b4-1a8f-47d9-ab6a-9cb85226cfb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:05.461229 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1db3b4-1a8f-47d9-ab6a-9cb85226cfb0 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:05.464962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 134/262] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:05.469996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 129/263] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:05.474750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebef77-a725-4870-be62-bfab1fa27131 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:05.475842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1db3b4-1a8f-47d9-ab6a-9cb85226cfb0 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:05.606033 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1db3b4-1a8f-47d9-ab6a-9cb85226cfb0 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:05.634203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebef77-a725-4870-be62-bfab1fa27131 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:05.657840 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0bd8d-7a38-4efb-8cbf-3bc4767408b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:05.659177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0bd8d-7a38-4efb-8cbf-3bc4767408b9 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:05.659693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0bd8d-7a38-4efb-8cbf-3bc4767408b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:05.660172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0bd8d-7a38-4efb-8cbf-3bc4767408b9 None admin] PATH_INFO: `/v3/users/fe40aadfa44049f58071a54c1101a548` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:05.661760 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0bd8d-7a38-4efb-8cbf-3bc4767408b9 None admin] RBAC: Authorizing `identity:get_user(user_id=fe40aadfa44049f58071a54c1101a548)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:05.663513 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0bd8d-7a38-4efb-8cbf-3bc4767408b9 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:05.667812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 130/264] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:05 2022] GET /identity/v3/users/fe40aadfa44049f58071a54c1101a548 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:05.674033 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0bd8d-7a38-4efb-8cbf-3bc4767408b9 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:05.723241 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2b8c0-3317-4a47-8d5e-db4ad5abb94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:05.724750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2b8c0-3317-4a47-8d5e-db4ad5abb94e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:05.725239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2b8c0-3317-4a47-8d5e-db4ad5abb94e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:05.725731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2b8c0-3317-4a47-8d5e-db4ad5abb94e None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:05.731981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2b8c0-3317-4a47-8d5e-db4ad5abb94e None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:05.733670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2b8c0-3317-4a47-8d5e-db4ad5abb94e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:05.736179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 135/265] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:05 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:05.742558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2b8c0-3317-4a47-8d5e-db4ad5abb94e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:05.849642 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0bd8d-7a38-4efb-8cbf-3bc4767408b9 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:05.887636 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5b9ae-4961-4538-be2b-17322061f5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:05.889310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5b9ae-4961-4538-be2b-17322061f5a8 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:05.894412 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5b9ae-4961-4538-be2b-17322061f5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:05.894962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5b9ae-4961-4538-be2b-17322061f5a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:05.949602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2b8c0-3317-4a47-8d5e-db4ad5abb94e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:06.017000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5b9ae-4961-4538-be2b-17322061f5a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:06.019570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5b9ae-4961-4538-be2b-17322061f5a8 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:06.020942 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4ebfd-72e6-4b67-8bdd-8f7595491823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:06.021595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4ebfd-72e6-4b67-8bdd-8f7595491823 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:06.021877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4ebfd-72e6-4b67-8bdd-8f7595491823 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:06.022154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4ebfd-72e6-4b67-8bdd-8f7595491823 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:06.023040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4ebfd-72e6-4b67-8bdd-8f7595491823 None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=a77cd5b44fbe457d82395d5a96360089, user.id=fe40aadfa44049f58071a54c1101a548) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:06.023352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4ebfd-72e6-4b67-8bdd-8f7595491823 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:06.024810 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4ebfd-72e6-4b67-8bdd-8f7595491823 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:06.031465 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 136/266] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:05 2022] GET /identity/v3/role_assignments?user.id=fe40aadfa44049f58071a54c1101a548&scope.project.id=a77cd5b44fbe457d82395d5a96360089&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:06.136511 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 131/267] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:06.142040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4ebfd-72e6-4b67-8bdd-8f7595491823 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:06.288656 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4ebfd-72e6-4b67-8bdd-8f7595491823 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:06.339734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442ab14-be71-4881-93f6-5058db343591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:06.340594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442ab14-be71-4881-93f6-5058db343591 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:06.340883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442ab14-be71-4881-93f6-5058db343591 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:06.341157 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442ab14-be71-4881-93f6-5058db343591 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:06.347588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442ab14-be71-4881-93f6-5058db343591 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:06.349084 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442ab14-be71-4881-93f6-5058db343591 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:06.351233 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 137/268] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:06 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:06.356989 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5b9ae-4961-4538-be2b-17322061f5a8 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:06.472427 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5b9ae-4961-4538-be2b-17322061f5a8 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:06.510402 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7a83c-3631-45c7-a3a9-454496763b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:06.511445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7a83c-3631-45c7-a3a9-454496763b65 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:06.511926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7a83c-3631-45c7-a3a9-454496763b65 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:06.512399 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7a83c-3631-45c7-a3a9-454496763b65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:06.513421 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7a83c-3631-45c7-a3a9-454496763b65 None admin] RBAC: Adding query filter params (role.id=a29f4364896c46dd84dcee5b969b9898, scope.project.id=a77cd5b44fbe457d82395d5a96360089, user.id=fe40aadfa44049f58071a54c1101a548) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:06.513936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7a83c-3631-45c7-a3a9-454496763b65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:06.516276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7a83c-3631-45c7-a3a9-454496763b65 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:06.523312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 132/269] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:06 2022] GET /identity/v3/role_assignments?user.id=fe40aadfa44049f58071a54c1101a548&scope.project.id=a77cd5b44fbe457d82395d5a96360089&role.id=a29f4364896c46dd84dcee5b969b9898 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:07.190227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee4575-6f7c-413e-b478-9ce1c1957cca None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:07.190227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee4575-6f7c-413e-b478-9ce1c1957cca None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:07.190227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee4575-6f7c-413e-b478-9ce1c1957cca None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:07.191134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 138/270] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:07.196456 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868678bf-01b4-4413-ac6d-121a7705565c None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:07.196856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868678bf-01b4-4413-ac6d-121a7705565c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:07.197037 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868678bf-01b4-4413-ac6d-121a7705565c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:07.217355 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-868678bf-01b4-4413-ac6d-121a7705565c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:07.218576 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-868678bf-01b4-4413-ac6d-121a7705565c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:07.262540 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea187a-b77a-42b9-a48a-5a50d4001a65 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:07.262956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea187a-b77a-42b9-a48a-5a50d4001a65 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:07.263239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea187a-b77a-42b9-a48a-5a50d4001a65 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:07.264772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 139/271] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:07.271497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4712b5-1c09-415c-ae23-0f1e9fae04f1 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:07.271781 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4712b5-1c09-415c-ae23-0f1e9fae04f1 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:07.272056 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4712b5-1c09-415c-ae23-0f1e9fae04f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:07.288930 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-2e4712b5-1c09-415c-ae23-0f1e9fae04f1 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:07.290119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2e4712b5-1c09-415c-ae23-0f1e9fae04f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:07.300836 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 133/272] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:07.303895 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f230ac-0c2e-4326-9b8b-a3538fa1a4d9 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:07.304334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f230ac-0c2e-4326-9b8b-a3538fa1a4d9 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:07.304754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f230ac-0c2e-4326-9b8b-a3538fa1a4d9 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:07.308717 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 134/273] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:07.315169 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c6ff8-63dd-444a-8788-e8a1ad9d036a None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:07.315592 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c6ff8-63dd-444a-8788-e8a1ad9d036a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:07.316197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c6ff8-63dd-444a-8788-e8a1ad9d036a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:07.338775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-631c6ff8-63dd-444a-8788-e8a1ad9d036a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:07.340230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-631c6ff8-63dd-444a-8788-e8a1ad9d036a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:07.380196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 140/274] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:07.381076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4712b5-1c09-415c-ae23-0f1e9fae04f1 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:07.411282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 135/275] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:07.412327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c6ff8-63dd-444a-8788-e8a1ad9d036a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:07.487748 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4712b5-1c09-415c-ae23-0f1e9fae04f1 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:07.513275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c6ff8-63dd-444a-8788-e8a1ad9d036a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:07.520004 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67576c6-bd6a-4aba-8998-01bb5b1d3151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:07.520792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67576c6-bd6a-4aba-8998-01bb5b1d3151 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:07.521075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67576c6-bd6a-4aba-8998-01bb5b1d3151 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:07.521347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67576c6-bd6a-4aba-8998-01bb5b1d3151 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:07.522508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67576c6-bd6a-4aba-8998-01bb5b1d3151 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:07.524001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67576c6-bd6a-4aba-8998-01bb5b1d3151 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:07.525636 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 141/276] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:07.526390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67576c6-bd6a-4aba-8998-01bb5b1d3151 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:07.545957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea971a-f141-44b7-a8ed-c6fee67a6c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:07.547299 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea971a-f141-44b7-a8ed-c6fee67a6c43 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:07.547579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea971a-f141-44b7-a8ed-c6fee67a6c43 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:07.547853 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea971a-f141-44b7-a8ed-c6fee67a6c43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:07.557693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea971a-f141-44b7-a8ed-c6fee67a6c43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-0cea971a-f141-44b7-a8ed-c6fee67a6c43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:07.565901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:07.566952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:07.567779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:07.567779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:07.567779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:07.567779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:07.567779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:07.567779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:07.567779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:07.567779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:07.567779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:07.567779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:07.567779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:07.567779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:07.567779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 136/277] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:07.572238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc9673-bad0-4af3-be83-b9e5a3a263ae None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:07.572532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc9673-bad0-4af3-be83-b9e5a3a263ae None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:07.572808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc9673-bad0-4af3-be83-b9e5a3a263ae None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:07.574099 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 137/278] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:07.574824 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc9673-bad0-4af3-be83-b9e5a3a263ae None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:07.620965 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67576c6-bd6a-4aba-8998-01bb5b1d3151 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:07.660169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475f356-9885-4653-9df1-a27572f158f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:07.660957 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475f356-9885-4653-9df1-a27572f158f8 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:07.661232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475f356-9885-4653-9df1-a27572f158f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:07.661548 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475f356-9885-4653-9df1-a27572f158f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:07.671157 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475f356-9885-4653-9df1-a27572f158f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-d475f356-9885-4653-9df1-a27572f158f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:07.679198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:07.680134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:07.686050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:07.686050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:07.686050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:07.686050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:07.686050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:07.686050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:07.686050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:07.686050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:07.686050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:07.686050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:07.686050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:07.686050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:07.686673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc9673-bad0-4af3-be83-b9e5a3a263ae None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:07.686772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 142/279] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:07.686772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475f356-9885-4653-9df1-a27572f158f8 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:07.711371 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67e76b-ef21-4425-8db8-49cac54a61c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:07.712029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e76b-ef21-4425-8db8-49cac54a61c1 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:07.712311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e76b-ef21-4425-8db8-49cac54a61c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:07.712589 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e76b-ef21-4425-8db8-49cac54a61c1 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:07.713287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67e76b-ef21-4425-8db8-49cac54a61c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:07.714757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67e76b-ef21-4425-8db8-49cac54a61c1 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:07.735262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 138/280] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:38:07 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:38:07.736637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127953c-4a5a-4ab9-80fc-11dfad80a690 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:07.736932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127953c-4a5a-4ab9-80fc-11dfad80a690 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:07.737214 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127953c-4a5a-4ab9-80fc-11dfad80a690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:07.754294 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-c127953c-4a5a-4ab9-80fc-11dfad80a690 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:07.755568 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c127953c-4a5a-4ab9-80fc-11dfad80a690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:07.780108 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475f356-9885-4653-9df1-a27572f158f8 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:07.815551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e31f51-ff63-4f5f-a9e1-ca837d1ca10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:07.816233 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e31f51-ff63-4f5f-a9e1-ca837d1ca10f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:07.816518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e31f51-ff63-4f5f-a9e1-ca837d1ca10f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:07.816795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e31f51-ff63-4f5f-a9e1-ca837d1ca10f None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:07.817547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e31f51-ff63-4f5f-a9e1-ca837d1ca10f None admin] RBAC: Adding query filter params (name=member) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:07.817857 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e31f51-ff63-4f5f-a9e1-ca837d1ca10f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:07.819330 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e31f51-ff63-4f5f-a9e1-ca837d1ca10f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:07.826293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 139/281] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:07.827182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127953c-4a5a-4ab9-80fc-11dfad80a690 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:07.841542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 143/282] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:07.847883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e31f51-ff63-4f5f-a9e1-ca837d1ca10f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:07.943603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127953c-4a5a-4ab9-80fc-11dfad80a690 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:07.969376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e31f51-ff63-4f5f-a9e1-ca837d1ca10f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:07.981073 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a0cb4-bbc7-45da-8a58-cfa79e624dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:07.981744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a0cb4-bbc7-45da-8a58-cfa79e624dad None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:07.982020 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a0cb4-bbc7-45da-8a58-cfa79e624dad None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:07.982289 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a0cb4-bbc7-45da-8a58-cfa79e624dad None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:07.983127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a0cb4-bbc7-45da-8a58-cfa79e624dad None admin] RBAC: Adding query filter params (name=member) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:07.983427 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a0cb4-bbc7-45da-8a58-cfa79e624dad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:07.986387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a0cb4-bbc7-45da-8a58-cfa79e624dad None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:07.993772 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 140/283] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:07.994596 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a0cb4-bbc7-45da-8a58-cfa79e624dad None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:08.021339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5d343-23a2-41a7-9520-b497dbc4c514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:08.022311 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5d343-23a2-41a7-9520-b497dbc4c514 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:08.022664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5d343-23a2-41a7-9520-b497dbc4c514 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:08.023191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5d343-23a2-41a7-9520-b497dbc4c514 None admin] PATH_INFO: `/v3/users/723f9af578c44d16a23c8dd395d19b38` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:08.025326 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5d343-23a2-41a7-9520-b497dbc4c514 None admin] RBAC: Authorizing `identity:get_user(user_id=723f9af578c44d16a23c8dd395d19b38)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:08.026817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5d343-23a2-41a7-9520-b497dbc4c514 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:08.028485 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 144/284] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:07 2022] GET /identity/v3/users/723f9af578c44d16a23c8dd395d19b38 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:08.029253 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5d343-23a2-41a7-9520-b497dbc4c514 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:08.135090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a0cb4-bbc7-45da-8a58-cfa79e624dad None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:08.161976 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5d343-23a2-41a7-9520-b497dbc4c514 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:08.174435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bbb88-bbf2-476c-b874-88df536313ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:08.175256 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bbb88-bbf2-476c-b874-88df536313ff None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:08.175536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bbb88-bbf2-476c-b874-88df536313ff None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:08.175811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bbb88-bbf2-476c-b874-88df536313ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:08.186194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bbb88-bbf2-476c-b874-88df536313ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-945bbb88-bbf2-476c-b874-88df536313ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:08.200090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:08.202918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:08.204992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:08.204992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:08.204992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:08.204992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:08.204992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:08.204992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:08.204992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:08.204992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:08.204992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:08.204992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:08.204992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:38:08.204992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:08.206455 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f42b0-56fe-41ed-a9f6-684438717072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:08.207017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 141/285] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:08.207017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bbb88-bbf2-476c-b874-88df536313ff None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:08.207281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f42b0-56fe-41ed-a9f6-684438717072 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:08.207281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f42b0-56fe-41ed-a9f6-684438717072 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:08.207281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f42b0-56fe-41ed-a9f6-684438717072 None admin] PATH_INFO: `/v3/users/f20d0c5b470a4e43aeaea06f882830f4` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:08.216983 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f42b0-56fe-41ed-a9f6-684438717072 None admin] RBAC: Authorizing `identity:get_user(user_id=f20d0c5b470a4e43aeaea06f882830f4)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:08.218509 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f42b0-56fe-41ed-a9f6-684438717072 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:08.225371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 145/286] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:08 2022] GET /identity/v3/users/f20d0c5b470a4e43aeaea06f882830f4 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:08.226037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f42b0-56fe-41ed-a9f6-684438717072 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:08.309820 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bbb88-bbf2-476c-b874-88df536313ff None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:08.341261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f42b0-56fe-41ed-a9f6-684438717072 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:08.361300 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ba147-9fae-41fe-ae9f-5a85af35be4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:08.362328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ba147-9fae-41fe-ae9f-5a85af35be4d None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:08.362612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ba147-9fae-41fe-ae9f-5a85af35be4d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:08.362933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ba147-9fae-41fe-ae9f-5a85af35be4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:08.388879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0726669-b0b1-456d-80c5-6beaf38fc1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:08.389742 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0726669-b0b1-456d-80c5-6beaf38fc1bc None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:08.390022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0726669-b0b1-456d-80c5-6beaf38fc1bc None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:08.390377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0726669-b0b1-456d-80c5-6beaf38fc1bc None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:08.391223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0726669-b0b1-456d-80c5-6beaf38fc1bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:08.391526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0726669-b0b1-456d-80c5-6beaf38fc1bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:08.392913 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0726669-b0b1-456d-80c5-6beaf38fc1bc None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:08.401475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 146/287] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:08.402554 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0726669-b0b1-456d-80c5-6beaf38fc1bc None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:08.448859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ba147-9fae-41fe-ae9f-5a85af35be4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:08.450884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ba147-9fae-41fe-ae9f-5a85af35be4d None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:08.510104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0726669-b0b1-456d-80c5-6beaf38fc1bc None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:08.545171 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac3858-b448-49bc-8daa-f317577b5a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:08.546171 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac3858-b448-49bc-8daa-f317577b5a44 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:08.547569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac3858-b448-49bc-8daa-f317577b5a44 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:08.548082 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac3858-b448-49bc-8daa-f317577b5a44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:08.549154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac3858-b448-49bc-8daa-f317577b5a44 None admin] RBAC: Adding query filter params (role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d, user.id=f20d0c5b470a4e43aeaea06f882830f4, scope.system=all) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:08.549728 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac3858-b448-49bc-8daa-f317577b5a44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:08.551434 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac3858-b448-49bc-8daa-f317577b5a44 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:08.556736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 147/288] 10.222.0.35 () {56 vars in 1308 bytes} [Fri May 13 21:38:08 2022] GET /identity/v3/role_assignments?user.id=f20d0c5b470a4e43aeaea06f882830f4&scope.system=all&role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:08.558044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac3858-b448-49bc-8daa-f317577b5a44 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:08.574744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 142/289] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:08.578039 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ba147-9fae-41fe-ae9f-5a85af35be4d None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:08.768776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac3858-b448-49bc-8daa-f317577b5a44 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:08.792174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ba147-9fae-41fe-ae9f-5a85af35be4d None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:08.821115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a70554-c6c6-451b-ac9a-29aa61d311f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:08.821985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a70554-c6c6-451b-ac9a-29aa61d311f4 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:08.822270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a70554-c6c6-451b-ac9a-29aa61d311f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:08.822548 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a70554-c6c6-451b-ac9a-29aa61d311f4 None admin] PATH_INFO: `/v3/users/4894956f37c14da1ab943d20a4edcc4a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:08.823931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a70554-c6c6-451b-ac9a-29aa61d311f4 None admin] RBAC: Authorizing `identity:get_user(user_id=4894956f37c14da1ab943d20a4edcc4a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:08.825288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a70554-c6c6-451b-ac9a-29aa61d311f4 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:08.828343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 148/290] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:08 2022] GET /identity/v3/users/4894956f37c14da1ab943d20a4edcc4a => generated 297 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:08.833659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a70554-c6c6-451b-ac9a-29aa61d311f4 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:08.834396 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f066e2-d3a8-4b0b-ad68-50cad106341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:08.835514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f066e2-d3a8-4b0b-ad68-50cad106341b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:08.836157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f066e2-d3a8-4b0b-ad68-50cad106341b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:08.836627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f066e2-d3a8-4b0b-ad68-50cad106341b None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:08.838026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f066e2-d3a8-4b0b-ad68-50cad106341b None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:08.839756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f066e2-d3a8-4b0b-ad68-50cad106341b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:08.841651 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 143/291] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:08 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:08.848040 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f066e2-d3a8-4b0b-ad68-50cad106341b None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:08.946943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f066e2-d3a8-4b0b-ad68-50cad106341b None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:08.981772 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2243e-1aea-40f1-88a8-ecdc618cabff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:08.982844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2243e-1aea-40f1-88a8-ecdc618cabff None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:08.983328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2243e-1aea-40f1-88a8-ecdc618cabff None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:08.983826 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2243e-1aea-40f1-88a8-ecdc618cabff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:08.984853 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2243e-1aea-40f1-88a8-ecdc618cabff None admin] RBAC: Adding query filter params (role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d, scope.project.id=a77cd5b44fbe457d82395d5a96360089, user.id=723f9af578c44d16a23c8dd395d19b38) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:08.985386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2243e-1aea-40f1-88a8-ecdc618cabff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:08.987073 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2243e-1aea-40f1-88a8-ecdc618cabff None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:08.993658 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 144/292] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:08 2022] GET /identity/v3/role_assignments?user.id=723f9af578c44d16a23c8dd395d19b38&scope.project.id=a77cd5b44fbe457d82395d5a96360089&role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:09.024771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a70554-c6c6-451b-ac9a-29aa61d311f4 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:09.076502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3898769-4ba2-4658-907b-2d2b37a86b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:09.077401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3898769-4ba2-4658-907b-2d2b37a86b73 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:09.078254 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3898769-4ba2-4658-907b-2d2b37a86b73 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:09.078538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3898769-4ba2-4658-907b-2d2b37a86b73 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:09.079868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3898769-4ba2-4658-907b-2d2b37a86b73 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:09.081370 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3898769-4ba2-4658-907b-2d2b37a86b73 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:09.083578 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 149/293] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:08 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:09.088966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2243e-1aea-40f1-88a8-ecdc618cabff None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:09.263868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2243e-1aea-40f1-88a8-ecdc618cabff None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:09.307169 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783fa06-e824-4fa4-aa4d-aeb9a062998c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:09.307976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783fa06-e824-4fa4-aa4d-aeb9a062998c None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:09.308254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783fa06-e824-4fa4-aa4d-aeb9a062998c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:09.308525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783fa06-e824-4fa4-aa4d-aeb9a062998c None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089/users/4894956f37c14da1ab943d20a4edcc4a/roles/94938baf3b7443bd8e53ce11902c7247` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:09.313281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783fa06-e824-4fa4-aa4d-aeb9a062998c None admin] RBAC: Authorizing `identity:create_grant(project_id=a77cd5b44fbe457d82395d5a96360089, user_id=4894956f37c14da1ab943d20a4edcc4a, role_id=94938baf3b7443bd8e53ce11902c7247)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:09.314754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783fa06-e824-4fa4-aa4d-aeb9a062998c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:09.328922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 145/294] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:38:09 2022] PUT /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089/users/4894956f37c14da1ab943d20a4edcc4a/roles/94938baf3b7443bd8e53ce11902c7247 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:38:10.219752 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5323fb3-6755-49b7-a368-0bccb1208d22 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:10.220277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5323fb3-6755-49b7-a368-0bccb1208d22 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:10.220513 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5323fb3-6755-49b7-a368-0bccb1208d22 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:10.221977 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 150/295] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:10.227297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ec93c-a043-468a-ac8a-23f04e5c7d56 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:10.227690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ec93c-a043-468a-ac8a-23f04e5c7d56 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:10.228065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ec93c-a043-468a-ac8a-23f04e5c7d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:10.245214 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-9c5ec93c-a043-468a-ac8a-23f04e5c7d56 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:10.246451 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9c5ec93c-a043-468a-ac8a-23f04e5c7d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:10.305956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f3792-efe7-4780-b6ad-409e1419689d None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:10.306295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f3792-efe7-4780-b6ad-409e1419689d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:10.306527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f3792-efe7-4780-b6ad-409e1419689d None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:10.308194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 151/296] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:10.313775 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e071da-04ce-4588-bfab-10d208876ab4 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:10.314066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e071da-04ce-4588-bfab-10d208876ab4 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:10.314349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e071da-04ce-4588-bfab-10d208876ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:10.318059 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 146/297] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:10.330835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-a3e071da-04ce-4588-bfab-10d208876ab4 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:10.332300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a3e071da-04ce-4588-bfab-10d208876ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:10.339852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ec93c-a043-468a-ac8a-23f04e5c7d56 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:10.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 152/298] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:10.428136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e071da-04ce-4588-bfab-10d208876ab4 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:10.460594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ec93c-a043-468a-ac8a-23f04e5c7d56 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:10.504548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65509d9-0a91-403b-a2bd-3f787b72df08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:10.505378 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65509d9-0a91-403b-a2bd-3f787b72df08 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:10.505658 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65509d9-0a91-403b-a2bd-3f787b72df08 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:10.505936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65509d9-0a91-403b-a2bd-3f787b72df08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:10.508067 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65509d9-0a91-403b-a2bd-3f787b72df08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:10.510000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65509d9-0a91-403b-a2bd-3f787b72df08 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:10.516827 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 147/299] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:10.518588 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65509d9-0a91-403b-a2bd-3f787b72df08 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:10.534016 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e071da-04ce-4588-bfab-10d208876ab4 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:10.571623 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c46a8-a42e-4d60-ae03-840837ac9f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:10.572452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c46a8-a42e-4d60-ae03-840837ac9f69 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:10.572736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c46a8-a42e-4d60-ae03-840837ac9f69 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:10.573011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c46a8-a42e-4d60-ae03-840837ac9f69 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:10.583730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c46a8-a42e-4d60-ae03-840837ac9f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-696c46a8-a42e-4d60-ae03-840837ac9f69 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:10.591655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:10.592714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:10.594051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:10.594051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:10.594051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:10.594051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:10.594051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:10.594051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:10.594051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:10.594051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:10.594051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:10.594051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:10.594051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:38:10.594051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:10.596739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 153/300] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:10.604079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c46a8-a42e-4d60-ae03-840837ac9f69 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:10.638277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65509d9-0a91-403b-a2bd-3f787b72df08 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:10.673431 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84597d17-d0ff-4be8-874e-cf2002b8fe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:10.674339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84597d17-d0ff-4be8-874e-cf2002b8fe81 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:10.674798 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84597d17-d0ff-4be8-874e-cf2002b8fe81 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:10.675287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84597d17-d0ff-4be8-874e-cf2002b8fe81 None admin] PATH_INFO: `/v3/groups` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:10.676169 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84597d17-d0ff-4be8-874e-cf2002b8fe81 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:10.677798 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84597d17-d0ff-4be8-874e-cf2002b8fe81 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:10.696035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 148/301] 10.222.0.35 () {60 vars in 1134 bytes} [Fri May 13 21:38:10 2022] POST /identity/v3/groups => generated 229 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:38:10.715998 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c46a8-a42e-4d60-ae03-840837ac9f69 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:10.756100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5feccad-bfc5-435a-a2f9-d09d7ba3575b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:10.757169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5feccad-bfc5-435a-a2f9-d09d7ba3575b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:10.757655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5feccad-bfc5-435a-a2f9-d09d7ba3575b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:10.758181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5feccad-bfc5-435a-a2f9-d09d7ba3575b None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:10.759213 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5feccad-bfc5-435a-a2f9-d09d7ba3575b None admin] RBAC: Adding query filter params (name=reader) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:10.759738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5feccad-bfc5-435a-a2f9-d09d7ba3575b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:10.761384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5feccad-bfc5-435a-a2f9-d09d7ba3575b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:10.771550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 154/302] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:10.779922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84597d17-d0ff-4be8-874e-cf2002b8fe81 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:10.889715 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84597d17-d0ff-4be8-874e-cf2002b8fe81 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:10.928942 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58f7bb-a678-4e5c-b67a-8913a65e48a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:10.929743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58f7bb-a678-4e5c-b67a-8913a65e48a6 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:10.930026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58f7bb-a678-4e5c-b67a-8913a65e48a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:10.930300 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58f7bb-a678-4e5c-b67a-8913a65e48a6 None admin] PATH_INFO: `/v3/users/0c32fab34ff643a38d9eb1ee975f4c25` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:10.945683 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58f7bb-a678-4e5c-b67a-8913a65e48a6 None admin] RBAC: Authorizing `identity:get_user(user_id=0c32fab34ff643a38d9eb1ee975f4c25)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:10.947233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58f7bb-a678-4e5c-b67a-8913a65e48a6 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:10.949197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 149/303] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:10 2022] GET /identity/v3/users/0c32fab34ff643a38d9eb1ee975f4c25 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:10.956101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5feccad-bfc5-435a-a2f9-d09d7ba3575b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:11.056617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09d454-3e15-42dc-a02a-7205ff49d16e None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:11.057025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09d454-3e15-42dc-a02a-7205ff49d16e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:11.057313 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09d454-3e15-42dc-a02a-7205ff49d16e None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:11.058532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5feccad-bfc5-435a-a2f9-d09d7ba3575b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:11.059023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 150/304] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:11.068835 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c0a3d-92e5-43c5-801f-69c38c06f901 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:11.069244 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c0a3d-92e5-43c5-801f-69c38c06f901 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:11.069625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c0a3d-92e5-43c5-801f-69c38c06f901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:11.090195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-b72c0a3d-92e5-43c5-801f-69c38c06f901 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:11.092271 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b72c0a3d-92e5-43c5-801f-69c38c06f901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:11.095939 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4b3f9-2c95-4760-9bd3-da97cebb72bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:11.096590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4b3f9-2c95-4760-9bd3-da97cebb72bb None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:11.096867 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4b3f9-2c95-4760-9bd3-da97cebb72bb None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:11.097139 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4b3f9-2c95-4760-9bd3-da97cebb72bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:11.098021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4b3f9-2c95-4760-9bd3-da97cebb72bb None admin] RBAC: Adding query filter params (role.id=94938baf3b7443bd8e53ce11902c7247, user.id=0c32fab34ff643a38d9eb1ee975f4c25, scope.system=all) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:11.098334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4b3f9-2c95-4760-9bd3-da97cebb72bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:11.099837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4b3f9-2c95-4760-9bd3-da97cebb72bb None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:11.105046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 155/305] 10.222.0.35 () {56 vars in 1308 bytes} [Fri May 13 21:38:10 2022] GET /identity/v3/role_assignments?user.id=0c32fab34ff643a38d9eb1ee975f4c25&scope.system=all&role.id=94938baf3b7443bd8e53ce11902c7247 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:11.168551 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 151/306] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:11.192473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4b3f9-2c95-4760-9bd3-da97cebb72bb None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:11.304939 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4b3f9-2c95-4760-9bd3-da97cebb72bb None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:11.338428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02464a71-2078-4890-81c3-e11e42ed3dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:11.339337 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02464a71-2078-4890-81c3-e11e42ed3dd9 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:11.339623 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02464a71-2078-4890-81c3-e11e42ed3dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:11.339944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02464a71-2078-4890-81c3-e11e42ed3dd9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:11.350853 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02464a71-2078-4890-81c3-e11e42ed3dd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-02464a71-2078-4890-81c3-e11e42ed3dd9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:11.359762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:11.360653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:11.361468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:11.361468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:11.361468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:11.361468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:11.361468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:11.361468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:11.361468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:11.361468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:11.361468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:11.361468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:11.361468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:11.361468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:11.361468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 156/307] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:11.367115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c0a3d-92e5-43c5-801f-69c38c06f901 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:11.470343 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c0a3d-92e5-43c5-801f-69c38c06f901 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:11.510721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d54acd-4a32-463a-8102-4ad1ea887033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:11.511398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d54acd-4a32-463a-8102-4ad1ea887033 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:11.511676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d54acd-4a32-463a-8102-4ad1ea887033 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:11.511946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d54acd-4a32-463a-8102-4ad1ea887033 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:11.512693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d54acd-4a32-463a-8102-4ad1ea887033 None admin] RBAC: Adding query filter params (name=member) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:11.512995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d54acd-4a32-463a-8102-4ad1ea887033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:11.514370 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d54acd-4a32-463a-8102-4ad1ea887033 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:11.523363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 152/308] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:11.528587 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02464a71-2078-4890-81c3-e11e42ed3dd9 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:11.635175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02464a71-2078-4890-81c3-e11e42ed3dd9 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:11.652678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bbce6-4520-4bcb-a9e9-dd568117d4b0 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:11.652980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bbce6-4520-4bcb-a9e9-dd568117d4b0 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:11.653239 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bbce6-4520-4bcb-a9e9-dd568117d4b0 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:11.654758 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 153/309] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:11.661906 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271909d-9369-4046-9ef6-f903ba8e6833 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:11.662188 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271909d-9369-4046-9ef6-f903ba8e6833 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:11.662464 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271909d-9369-4046-9ef6-f903ba8e6833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:11.672195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fdda7-23a0-4d0b-83d6-b0b969c102f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:11.673108 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fdda7-23a0-4d0b-83d6-b0b969c102f7 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:11.673389 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fdda7-23a0-4d0b-83d6-b0b969c102f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:11.673661 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fdda7-23a0-4d0b-83d6-b0b969c102f7 None admin] PATH_INFO: `/v3/users/f20d0c5b470a4e43aeaea06f882830f4` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:11.674896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fdda7-23a0-4d0b-83d6-b0b969c102f7 None admin] RBAC: Authorizing `identity:get_user(user_id=f20d0c5b470a4e43aeaea06f882830f4)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:11.676357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fdda7-23a0-4d0b-83d6-b0b969c102f7 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:11.678779 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 157/310] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:11 2022] GET /identity/v3/users/f20d0c5b470a4e43aeaea06f882830f4 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:11.682524 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7271909d-9369-4046-9ef6-f903ba8e6833 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:11.683934 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7271909d-9369-4046-9ef6-f903ba8e6833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:11.684373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fdda7-23a0-4d0b-83d6-b0b969c102f7 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:11.752773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 154/311] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:11.775597 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271909d-9369-4046-9ef6-f903ba8e6833 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:11.810872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fdda7-23a0-4d0b-83d6-b0b969c102f7 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:11.852869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9097f12-7db7-45db-baf1-370fca5b7098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:11.854381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9097f12-7db7-45db-baf1-370fca5b7098 None admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:11.855164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9097f12-7db7-45db-baf1-370fca5b7098 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:11.855661 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9097f12-7db7-45db-baf1-370fca5b7098 None admin] PATH_INFO: `/v3/system/users/f20d0c5b470a4e43aeaea06f882830f4/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:11.858058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9097f12-7db7-45db-baf1-370fca5b7098 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f20d0c5b470a4e43aeaea06f882830f4, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:11.860478 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9097f12-7db7-45db-baf1-370fca5b7098 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:11.872518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 158/312] 10.222.0.35 () {58 vars in 1334 bytes} [Fri May 13 21:38:11 2022] PUT /identity/v3/system/users/f20d0c5b470a4e43aeaea06f882830f4/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:38:11.883020 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271909d-9369-4046-9ef6-f903ba8e6833 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:11.920875 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8cda6-cdaf-41b1-8f84-7bc69621fae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:11.922538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8cda6-cdaf-41b1-8f84-7bc69621fae6 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:11.922995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8cda6-cdaf-41b1-8f84-7bc69621fae6 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:11.923408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8cda6-cdaf-41b1-8f84-7bc69621fae6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:11.935897 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8cda6-cdaf-41b1-8f84-7bc69621fae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-48e8cda6-cdaf-41b1-8f84-7bc69621fae6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:11.944749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:11.945881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:11.946884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:11.946884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:11.946884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:11.946884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:11.946884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:11.946884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:11.946884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:11.946884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:11.946884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:11.946884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:11.946884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:38:11.946884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:11.949363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 155/313] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:11.955558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9097f12-7db7-45db-baf1-370fca5b7098 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:12.066203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9097f12-7db7-45db-baf1-370fca5b7098 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:12.099798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fe831-db05-4fc9-aa80-2a689a57c72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:12.100467 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fe831-db05-4fc9-aa80-2a689a57c72a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:12.100750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fe831-db05-4fc9-aa80-2a689a57c72a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:12.101027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fe831-db05-4fc9-aa80-2a689a57c72a None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:12.101767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fe831-db05-4fc9-aa80-2a689a57c72a None admin] RBAC: Adding query filter params (name=reader) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:12.102071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fe831-db05-4fc9-aa80-2a689a57c72a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:12.103494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fe831-db05-4fc9-aa80-2a689a57c72a None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:12.112754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 159/314] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:12.118223 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8cda6-cdaf-41b1-8f84-7bc69621fae6 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:12.228946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8cda6-cdaf-41b1-8f84-7bc69621fae6 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:12.263102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792c871-e823-4c4f-ac9c-0296a28615f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:12.263967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792c871-e823-4c4f-ac9c-0296a28615f6 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:12.264249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792c871-e823-4c4f-ac9c-0296a28615f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:12.264533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792c871-e823-4c4f-ac9c-0296a28615f6 None admin] PATH_INFO: `/v3/users/4894956f37c14da1ab943d20a4edcc4a` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:12.266032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792c871-e823-4c4f-ac9c-0296a28615f6 None admin] RBAC: Authorizing `identity:get_user(user_id=4894956f37c14da1ab943d20a4edcc4a)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:12.267499 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792c871-e823-4c4f-ac9c-0296a28615f6 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:12.269474 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 156/315] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:12 2022] GET /identity/v3/users/4894956f37c14da1ab943d20a4edcc4a => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:12.274599 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fe831-db05-4fc9-aa80-2a689a57c72a None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:12.377543 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fe831-db05-4fc9-aa80-2a689a57c72a None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:12.412097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18e742-d061-4693-a6a0-38f439910406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:12.412742 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18e742-d061-4693-a6a0-38f439910406 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:12.413023 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18e742-d061-4693-a6a0-38f439910406 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:12.413296 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18e742-d061-4693-a6a0-38f439910406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:12.485910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18e742-d061-4693-a6a0-38f439910406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:12.487395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18e742-d061-4693-a6a0-38f439910406 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:12.584756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 160/316] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:12.589820 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792c871-e823-4c4f-ac9c-0296a28615f6 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:12.689094 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792c871-e823-4c4f-ac9c-0296a28615f6 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:12.723039 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f86fc7-8f57-416c-b281-463f004ab4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:12.723913 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f86fc7-8f57-416c-b281-463f004ab4e9 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:12.724194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f86fc7-8f57-416c-b281-463f004ab4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:12.724465 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f86fc7-8f57-416c-b281-463f004ab4e9 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:12.725663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f86fc7-8f57-416c-b281-463f004ab4e9 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:12.727152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f86fc7-8f57-416c-b281-463f004ab4e9 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:12.729044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 157/317] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:12 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:12.734654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18e742-d061-4693-a6a0-38f439910406 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:12.837357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18e742-d061-4693-a6a0-38f439910406 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:12.870444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139a52b-2f7a-4dba-995b-6dbacd95d138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:12.871118 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139a52b-2f7a-4dba-995b-6dbacd95d138 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:12.871399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139a52b-2f7a-4dba-995b-6dbacd95d138 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:12.871672 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139a52b-2f7a-4dba-995b-6dbacd95d138 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:12.872491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139a52b-2f7a-4dba-995b-6dbacd95d138 None admin] RBAC: Adding query filter params (role.id=94938baf3b7443bd8e53ce11902c7247, scope.project.id=a77cd5b44fbe457d82395d5a96360089, user.id=4894956f37c14da1ab943d20a4edcc4a) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:12.872797 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139a52b-2f7a-4dba-995b-6dbacd95d138 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:12.874818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139a52b-2f7a-4dba-995b-6dbacd95d138 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:12.881258 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 161/318] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:12 2022] GET /identity/v3/role_assignments?user.id=4894956f37c14da1ab943d20a4edcc4a&scope.project.id=a77cd5b44fbe457d82395d5a96360089&role.id=94938baf3b7443bd8e53ce11902c7247 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:13.134510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb9dbc-a888-41a5-ae21-1c8af8961afe None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:13.135043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb9dbc-a888-41a5-ae21-1c8af8961afe None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:13.135470 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb9dbc-a888-41a5-ae21-1c8af8961afe None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:13.137257 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 158/319] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:13.143546 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4288203-3ab3-4df5-9185-4f5cc36d2393 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:13.144093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4288203-3ab3-4df5-9185-4f5cc36d2393 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:13.144607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4288203-3ab3-4df5-9185-4f5cc36d2393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:13.167065 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-b4288203-3ab3-4df5-9185-4f5cc36d2393 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:13.168287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b4288203-3ab3-4df5-9185-4f5cc36d2393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:13.232021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 162/320] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:13.255835 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb9dbc-a888-41a5-ae21-1c8af8961afe None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:13.359303 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb9dbc-a888-41a5-ae21-1c8af8961afe None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:13.392636 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c1855-e0fd-4a02-8238-ad2e8247aea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:13.393350 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c1855-e0fd-4a02-8238-ad2e8247aea3 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:13.393635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c1855-e0fd-4a02-8238-ad2e8247aea3 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:13.393914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c1855-e0fd-4a02-8238-ad2e8247aea3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:13.395091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c1855-e0fd-4a02-8238-ad2e8247aea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:13.396646 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c1855-e0fd-4a02-8238-ad2e8247aea3 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:13.398307 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 159/321] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:13.403730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4288203-3ab3-4df5-9185-4f5cc36d2393 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:13.443942 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3d9e0-1aef-4115-891c-68064dbe3157 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:13.444245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3d9e0-1aef-4115-891c-68064dbe3157 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:13.444500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3d9e0-1aef-4115-891c-68064dbe3157 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:13.445991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 160/322] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:13.450869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea530ce-3213-4cef-aa7f-58f3a3ad002c None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:13.451157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea530ce-3213-4cef-aa7f-58f3a3ad002c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:13.451431 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea530ce-3213-4cef-aa7f-58f3a3ad002c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:13.467313 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-2ea530ce-3213-4cef-aa7f-58f3a3ad002c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:13.468507 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-2ea530ce-3213-4cef-aa7f-58f3a3ad002c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:13.504130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4288203-3ab3-4df5-9185-4f5cc36d2393 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:13.535624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929103f-c42b-4dd5-882f-44fc9e903762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:13.535929 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 161/323] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:13.536290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929103f-c42b-4dd5-882f-44fc9e903762 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:13.536566 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929103f-c42b-4dd5-882f-44fc9e903762 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:13.536838 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929103f-c42b-4dd5-882f-44fc9e903762 None admin] PATH_INFO: `/v3/groups` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:13.537518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929103f-c42b-4dd5-882f-44fc9e903762 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:13.538993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929103f-c42b-4dd5-882f-44fc9e903762 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:13.556218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 163/324] 10.222.0.35 () {60 vars in 1134 bytes} [Fri May 13 21:38:13 2022] POST /identity/v3/groups => generated 226 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:38:13.560274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea530ce-3213-4cef-aa7f-58f3a3ad002c None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:13.664015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea530ce-3213-4cef-aa7f-58f3a3ad002c None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:13.698325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d9942-a03f-4d73-b0bb-3f2aef403f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:13.699157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d9942-a03f-4d73-b0bb-3f2aef403f4e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:13.699440 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d9942-a03f-4d73-b0bb-3f2aef403f4e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:13.699713 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d9942-a03f-4d73-b0bb-3f2aef403f4e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:13.711851 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d9942-a03f-4d73-b0bb-3f2aef403f4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-751d9942-a03f-4d73-b0bb-3f2aef403f4e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:13.720273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:13.721206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:13.721968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:13.721968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:13.721968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:13.721968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:13.721968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:13.721968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:13.721968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:13.721968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:13.721968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:13.721968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:13.721968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:38:13.721968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:13.721968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 162/325] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:13.728596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929103f-c42b-4dd5-882f-44fc9e903762 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:13.830215 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929103f-c42b-4dd5-882f-44fc9e903762 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:13.871430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a7eff-84b2-4987-9cac-c2ceabc9c507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:13.872101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a7eff-84b2-4987-9cac-c2ceabc9c507 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:13.872383 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a7eff-84b2-4987-9cac-c2ceabc9c507 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:13.872657 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a7eff-84b2-4987-9cac-c2ceabc9c507 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:13.873412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a7eff-84b2-4987-9cac-c2ceabc9c507 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:13.873716 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a7eff-84b2-4987-9cac-c2ceabc9c507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:13.875516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a7eff-84b2-4987-9cac-c2ceabc9c507 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:13.884846 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 164/326] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:13.890489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d9942-a03f-4d73-b0bb-3f2aef403f4e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:14.008820 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d9942-a03f-4d73-b0bb-3f2aef403f4e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:14.045632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7fe13-c28e-48ef-901a-290ba2dcb88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:14.046795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7fe13-c28e-48ef-901a-290ba2dcb88c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:14.047246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7fe13-c28e-48ef-901a-290ba2dcb88c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:14.047666 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7fe13-c28e-48ef-901a-290ba2dcb88c None admin] PATH_INFO: `/v3/users/0c32fab34ff643a38d9eb1ee975f4c25` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:14.049152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7fe13-c28e-48ef-901a-290ba2dcb88c None admin] RBAC: Authorizing `identity:get_user(user_id=0c32fab34ff643a38d9eb1ee975f4c25)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:14.050695 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7fe13-c28e-48ef-901a-290ba2dcb88c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:14.052476 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 163/327] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:13 2022] GET /identity/v3/users/0c32fab34ff643a38d9eb1ee975f4c25 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:14.057851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a7eff-84b2-4987-9cac-c2ceabc9c507 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:14.186524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a7eff-84b2-4987-9cac-c2ceabc9c507 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:14.207329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a5236-b6af-41ba-9c91-7bb511352e87 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:14.207627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a5236-b6af-41ba-9c91-7bb511352e87 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:14.207891 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a5236-b6af-41ba-9c91-7bb511352e87 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:14.209541 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 164/328] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:14.214355 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa2584-ecaf-45c7-9dbe-2d084f4a567c None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:14.214643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa2584-ecaf-45c7-9dbe-2d084f4a567c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:14.214958 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa2584-ecaf-45c7-9dbe-2d084f4a567c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:14.240840 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-f2aa2584-ecaf-45c7-9dbe-2d084f4a567c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:14.244876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f2aa2584-ecaf-45c7-9dbe-2d084f4a567c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:14.246075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91bd0b-b539-480b-ac1b-b69abfb075d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:14.246990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91bd0b-b539-480b-ac1b-b69abfb075d4 None admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:14.247511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91bd0b-b539-480b-ac1b-b69abfb075d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:14.247791 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91bd0b-b539-480b-ac1b-b69abfb075d4 None admin] PATH_INFO: `/v3/system/users/0c32fab34ff643a38d9eb1ee975f4c25/roles/94938baf3b7443bd8e53ce11902c7247` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:14.250294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91bd0b-b539-480b-ac1b-b69abfb075d4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0c32fab34ff643a38d9eb1ee975f4c25, role_id=94938baf3b7443bd8e53ce11902c7247)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:14.251935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91bd0b-b539-480b-ac1b-b69abfb075d4 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:14.263096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 165/329] 10.222.0.35 () {58 vars in 1334 bytes} [Fri May 13 21:38:14 2022] PUT /identity/v3/system/users/0c32fab34ff643a38d9eb1ee975f4c25/roles/94938baf3b7443bd8e53ce11902c7247 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:38:14.364238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 165/330] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:14.394279 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91bd0b-b539-480b-ac1b-b69abfb075d4 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:14.533650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91bd0b-b539-480b-ac1b-b69abfb075d4 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:14.572848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be57fd-6cfe-4643-84d7-23944be51aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:14.573647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be57fd-6cfe-4643-84d7-23944be51aa8 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:14.573929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be57fd-6cfe-4643-84d7-23944be51aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:14.574201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be57fd-6cfe-4643-84d7-23944be51aa8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:14.589593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be57fd-6cfe-4643-84d7-23944be51aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-c2be57fd-6cfe-4643-84d7-23944be51aa8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:14.598234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:14.599190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:14.599963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:14.599963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:14.599963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:14.599963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:14.599963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:14.599963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:14.599963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:14.599963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:14.599963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:14.599963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:14.599963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:14.599963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:14.604115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 166/331] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:14.609344 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa2584-ecaf-45c7-9dbe-2d084f4a567c None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:14.722899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa2584-ecaf-45c7-9dbe-2d084f4a567c None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:14.760661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34397e-b691-4923-aea7-ffbda3b4f319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:14.761329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34397e-b691-4923-aea7-ffbda3b4f319 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:14.761607 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34397e-b691-4923-aea7-ffbda3b4f319 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:14.761876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34397e-b691-4923-aea7-ffbda3b4f319 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:14.762625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34397e-b691-4923-aea7-ffbda3b4f319 None admin] RBAC: Adding query filter params (name=member) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:14.762957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34397e-b691-4923-aea7-ffbda3b4f319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:14.764452 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34397e-b691-4923-aea7-ffbda3b4f319 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:14.775756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 166/332] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:14.782281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be57fd-6cfe-4643-84d7-23944be51aa8 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:14.898005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be57fd-6cfe-4643-84d7-23944be51aa8 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:14.933682 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77746ad7-5334-49d1-b2f9-124081171899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:14.934582 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77746ad7-5334-49d1-b2f9-124081171899 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:14.934902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77746ad7-5334-49d1-b2f9-124081171899 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:14.935179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77746ad7-5334-49d1-b2f9-124081171899 None admin] PATH_INFO: `/v3/users/f20d0c5b470a4e43aeaea06f882830f4` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:14.936499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77746ad7-5334-49d1-b2f9-124081171899 None admin] RBAC: Authorizing `identity:get_user(user_id=f20d0c5b470a4e43aeaea06f882830f4)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:14.938053 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77746ad7-5334-49d1-b2f9-124081171899 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:14.942555 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 167/333] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:14 2022] GET /identity/v3/users/f20d0c5b470a4e43aeaea06f882830f4 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:14.945960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34397e-b691-4923-aea7-ffbda3b4f319 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:15.049448 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34397e-b691-4923-aea7-ffbda3b4f319 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:15.089292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d80775-21f3-489a-aeb1-afdee17193fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:15.089964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d80775-21f3-489a-aeb1-afdee17193fe None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:15.090249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d80775-21f3-489a-aeb1-afdee17193fe None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:15.090523 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d80775-21f3-489a-aeb1-afdee17193fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:15.091436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d80775-21f3-489a-aeb1-afdee17193fe None admin] RBAC: Adding query filter params (role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d, user.id=f20d0c5b470a4e43aeaea06f882830f4, scope.system=all) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:15.091884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d80775-21f3-489a-aeb1-afdee17193fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:15.095684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d80775-21f3-489a-aeb1-afdee17193fe None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:15.108529 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 167/334] 10.222.0.35 () {56 vars in 1308 bytes} [Fri May 13 21:38:14 2022] GET /identity/v3/role_assignments?user.id=f20d0c5b470a4e43aeaea06f882830f4&scope.system=all&role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 510 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:16.461492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7aa2a5-400f-43d6-9b26-ca114bf8475b None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:16.462546 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7aa2a5-400f-43d6-9b26-ca114bf8475b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:16.463067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7aa2a5-400f-43d6-9b26-ca114bf8475b None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:16.469968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 168/335] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:16.474252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79716d90-09ad-40f2-9a3d-13716e103a5a None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:16.474741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79716d90-09ad-40f2-9a3d-13716e103a5a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:16.474828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79716d90-09ad-40f2-9a3d-13716e103a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:16.503393 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-79716d90-09ad-40f2-9a3d-13716e103a5a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:16.504641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-79716d90-09ad-40f2-9a3d-13716e103a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:16.600439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 168/336] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:16.699016 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7aa2a5-400f-43d6-9b26-ca114bf8475b None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:16.890516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7aa2a5-400f-43d6-9b26-ca114bf8475b None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:16.939105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fab001-f5f9-42b2-9c51-10b439b95401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:16.943885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fab001-f5f9-42b2-9c51-10b439b95401 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:16.944425 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fab001-f5f9-42b2-9c51-10b439b95401 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:16.944915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fab001-f5f9-42b2-9c51-10b439b95401 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:16.968041 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fab001-f5f9-42b2-9c51-10b439b95401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-85fab001-f5f9-42b2-9c51-10b439b95401 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:16.976114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:16.977028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:16.977847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:16.977847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:16.977847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:16.977847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:16.977847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:16.977847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:16.977847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:16.977847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:16.977847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:16.977847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:16.977847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:16.977847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:16.977847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 169/337] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:38:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:16.985073 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79716d90-09ad-40f2-9a3d-13716e103a5a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:17.147131 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79716d90-09ad-40f2-9a3d-13716e103a5a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:17.201366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b522d6-1a7a-4500-a42d-d7aa57e04aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:17.202035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b522d6-1a7a-4500-a42d-d7aa57e04aa3 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:17.202356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b522d6-1a7a-4500-a42d-d7aa57e04aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:17.202631 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b522d6-1a7a-4500-a42d-d7aa57e04aa3 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:17.203423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b522d6-1a7a-4500-a42d-d7aa57e04aa3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:17.203734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b522d6-1a7a-4500-a42d-d7aa57e04aa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:17.205115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b522d6-1a7a-4500-a42d-d7aa57e04aa3 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:17.214000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 169/338] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:17.223847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fab001-f5f9-42b2-9c51-10b439b95401 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:17.329558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3ae78-d319-49af-a6cf-fef39cdfd781 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:17.329844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3ae78-d319-49af-a6cf-fef39cdfd781 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:17.330340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3ae78-d319-49af-a6cf-fef39cdfd781 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:17.331952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 170/339] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:17.337057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff777cc-6765-4858-934e-79dd67ae30d7 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:17.337861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff777cc-6765-4858-934e-79dd67ae30d7 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:17.338651 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff777cc-6765-4858-934e-79dd67ae30d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:17.365329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-dff777cc-6765-4858-934e-79dd67ae30d7 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:17.367082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-dff777cc-6765-4858-934e-79dd67ae30d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:17.372140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fab001-f5f9-42b2-9c51-10b439b95401 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:17.438273 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163122a1-0ea0-463a-bf08-4f1a7f17de8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:17.439186 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163122a1-0ea0-463a-bf08-4f1a7f17de8e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:17.439635 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163122a1-0ea0-463a-bf08-4f1a7f17de8e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:17.440081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163122a1-0ea0-463a-bf08-4f1a7f17de8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:17.476611 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 171/340] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:17.516552 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff777cc-6765-4858-934e-79dd67ae30d7 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:17.541840 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163122a1-0ea0-463a-bf08-4f1a7f17de8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:17.544703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163122a1-0ea0-463a-bf08-4f1a7f17de8e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:17.689761 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 170/341] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:17.699835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163122a1-0ea0-463a-bf08-4f1a7f17de8e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:17.734418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff777cc-6765-4858-934e-79dd67ae30d7 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:17.830679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf0e20-ca92-4e26-9240-00623d022612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:17.831507 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf0e20-ca92-4e26-9240-00623d022612 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:17.831787 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf0e20-ca92-4e26-9240-00623d022612 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:17.832062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf0e20-ca92-4e26-9240-00623d022612 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:17.834332 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163122a1-0ea0-463a-bf08-4f1a7f17de8e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:17.855334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf0e20-ca92-4e26-9240-00623d022612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-47cf0e20-ca92-4e26-9240-00623d022612 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:17.863026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:17.863896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:17.864705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:17.864705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:17.864705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:17.864705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:17.864705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:17.864705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:17.864705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:17.864705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:17.864705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:17.864705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:17.864705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:17.864705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:17.864705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 172/342] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:17.865828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d8f2a-a386-4687-9ef3-500d9106541b None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:17.865828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d8f2a-a386-4687-9ef3-500d9106541b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:17.865990 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d8f2a-a386-4687-9ef3-500d9106541b None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:17.870664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 173/343] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:17.881243 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d8f2a-a386-4687-9ef3-500d9106541b None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:17.908654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f7f77-d8f6-439f-b4cb-a8246433a6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:17.909519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f7f77-d8f6-439f-b4cb-a8246433a6af None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:17.909794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f7f77-d8f6-439f-b4cb-a8246433a6af None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:17.910064 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f7f77-d8f6-439f-b4cb-a8246433a6af None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:17.911399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f7f77-d8f6-439f-b4cb-a8246433a6af None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:17.912842 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f7f77-d8f6-439f-b4cb-a8246433a6af None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:17.914552 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 171/344] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:17 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:17.915722 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bfdfb-b8b6-4cdb-aa3f-6791891c8274 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:17.916006 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bfdfb-b8b6-4cdb-aa3f-6791891c8274 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:17.916258 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bfdfb-b8b6-4cdb-aa3f-6791891c8274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:17.947759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-840bfdfb-b8b6-4cdb-aa3f-6791891c8274 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:17.949175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-840bfdfb-b8b6-4cdb-aa3f-6791891c8274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:18.046533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d8f2a-a386-4687-9ef3-500d9106541b None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:18.057931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 172/345] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:18.059013 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bfdfb-b8b6-4cdb-aa3f-6791891c8274 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:18.110572 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8c924-73bc-43e9-8edf-41ad7fd7c4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:18.111508 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8c924-73bc-43e9-8edf-41ad7fd7c4ba None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:18.111959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8c924-73bc-43e9-8edf-41ad7fd7c4ba None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:18.112400 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8c924-73bc-43e9-8edf-41ad7fd7c4ba None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:18.113302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8c924-73bc-43e9-8edf-41ad7fd7c4ba None admin] RBAC: Adding query filter params (name=member) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:18.113774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8c924-73bc-43e9-8edf-41ad7fd7c4ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:18.115406 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8c924-73bc-43e9-8edf-41ad7fd7c4ba None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:18.124613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 174/346] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:18.126209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417afba-c286-4321-99df-1f6d9cecc368 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:18.126680 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417afba-c286-4321-99df-1f6d9cecc368 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:18.127161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417afba-c286-4321-99df-1f6d9cecc368 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:18.128869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 175/347] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:18.133747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417afba-c286-4321-99df-1f6d9cecc368 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:18.183100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bfdfb-b8b6-4cdb-aa3f-6791891c8274 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:18.226351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36380fa8-b031-40a2-ac00-4fdbe281381c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:18.227520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36380fa8-b031-40a2-ac00-4fdbe281381c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:18.227803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36380fa8-b031-40a2-ac00-4fdbe281381c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:18.228103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36380fa8-b031-40a2-ac00-4fdbe281381c None admin] PATH_INFO: `/v3/groups/7ea69f2d69684b569f82ad5ec713f402` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:18.235759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36380fa8-b031-40a2-ac00-4fdbe281381c None admin] RBAC: Authorizing `identity:get_group(group_id=7ea69f2d69684b569f82ad5ec713f402)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:18.237223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36380fa8-b031-40a2-ac00-4fdbe281381c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:18.239104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 173/348] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:18 2022] GET /identity/v3/groups/7ea69f2d69684b569f82ad5ec713f402 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:18.240039 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36380fa8-b031-40a2-ac00-4fdbe281381c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:18.253665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417afba-c286-4321-99df-1f6d9cecc368 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:18.295876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd40f6-3e5d-4450-bb77-62ae61e573a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:18.297252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd40f6-3e5d-4450-bb77-62ae61e573a7 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:18.297535 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd40f6-3e5d-4450-bb77-62ae61e573a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:18.297810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd40f6-3e5d-4450-bb77-62ae61e573a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:18.312684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd40f6-3e5d-4450-bb77-62ae61e573a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-8dbd40f6-3e5d-4450-bb77-62ae61e573a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:18.320354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:18.321436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:18.322246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:18.322246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:18.322246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:18.322246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:18.322246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:18.322246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:18.322246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:18.322246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:18.322246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:18.322246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:18.322246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:18.322246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:18.324298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 176/349] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:18.325783 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3525aa-500b-4113-8247-49409543d39e None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:18.326077 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3525aa-500b-4113-8247-49409543d39e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:18.326684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3525aa-500b-4113-8247-49409543d39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:18.344920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-be3525aa-500b-4113-8247-49409543d39e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:18.346077 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-be3525aa-500b-4113-8247-49409543d39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:18.355057 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36380fa8-b031-40a2-ac00-4fdbe281381c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:18.399522 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4b62e-4026-40a2-aca1-18dd3962abfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:18.400446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4b62e-4026-40a2-aca1-18dd3962abfd None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:18.400899 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4b62e-4026-40a2-aca1-18dd3962abfd None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:18.401747 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4b62e-4026-40a2-aca1-18dd3962abfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:18.443131 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 177/350] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:18.444727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687c512-2264-4a93-bda6-39efcdbb6a6e None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:18.445196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687c512-2264-4a93-bda6-39efcdbb6a6e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:18.445653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687c512-2264-4a93-bda6-39efcdbb6a6e None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:18.447222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 178/351] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:18.448127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687c512-2264-4a93-bda6-39efcdbb6a6e None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:18.509021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4b62e-4026-40a2-aca1-18dd3962abfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:18.510429 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4b62e-4026-40a2-aca1-18dd3962abfd None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:18.553543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687c512-2264-4a93-bda6-39efcdbb6a6e None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:18.590358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a19ed-b150-407f-868b-8a8e1ee038be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:18.590581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a19ed-b150-407f-868b-8a8e1ee038be None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:18.590882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a19ed-b150-407f-868b-8a8e1ee038be None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:18.591166 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a19ed-b150-407f-868b-8a8e1ee038be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:18.592005 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a19ed-b150-407f-868b-8a8e1ee038be None admin] RBAC: Adding query filter params (group.id=7ea69f2d69684b569f82ad5ec713f402, role.id=a29f4364896c46dd84dcee5b969b9898, scope.project.id=80047f4aaba64866971a8b97d21dee0a) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:18.592318 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a19ed-b150-407f-868b-8a8e1ee038be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:18.594336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a19ed-b150-407f-868b-8a8e1ee038be None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:18.600697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 179/352] 10.222.0.35 () {56 vars in 1376 bytes} [Fri May 13 21:38:18 2022] GET /identity/v3/role_assignments?group.id=7ea69f2d69684b569f82ad5ec713f402&scope.project.id=80047f4aaba64866971a8b97d21dee0a&role.id=a29f4364896c46dd84dcee5b969b9898 => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:18.601907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a19ed-b150-407f-868b-8a8e1ee038be None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:18.635255 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 174/353] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:18.637222 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36616134-15b7-4a7d-a612-882c13a9eeaa None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:18.637663 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36616134-15b7-4a7d-a612-882c13a9eeaa None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:18.638164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36616134-15b7-4a7d-a612-882c13a9eeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:18.661300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-36616134-15b7-4a7d-a612-882c13a9eeaa None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:18.662975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-36616134-15b7-4a7d-a612-882c13a9eeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:18.717280 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a19ed-b150-407f-868b-8a8e1ee038be None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:18.736981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 175/354] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:18.738237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36616134-15b7-4a7d-a612-882c13a9eeaa None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:18.753276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a614de-2b33-4daf-8b7b-355ed42888e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:18.754226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a614de-2b33-4daf-8b7b-355ed42888e3 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:18.754646 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a614de-2b33-4daf-8b7b-355ed42888e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:18.755085 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a614de-2b33-4daf-8b7b-355ed42888e3 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:18.756003 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a614de-2b33-4daf-8b7b-355ed42888e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:18.756478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a614de-2b33-4daf-8b7b-355ed42888e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:18.761968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a614de-2b33-4daf-8b7b-355ed42888e3 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:18.775543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 180/355] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:18.776392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a614de-2b33-4daf-8b7b-355ed42888e3 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:18.879570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36616134-15b7-4a7d-a612-882c13a9eeaa None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:18.904216 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a614de-2b33-4daf-8b7b-355ed42888e3 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:18.937464 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808663f-596f-431b-88c0-cd6b3b843a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:18.938283 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808663f-596f-431b-88c0-cd6b3b843a85 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:18.938561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808663f-596f-431b-88c0-cd6b3b843a85 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:18.938858 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808663f-596f-431b-88c0-cd6b3b843a85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:18.952567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ded42-1bdc-483f-a9be-79e783b85845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:18.953435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ded42-1bdc-483f-a9be-79e783b85845 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:18.953716 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ded42-1bdc-483f-a9be-79e783b85845 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:18.953936 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808663f-596f-431b-88c0-cd6b3b843a85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:18.954034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ded42-1bdc-483f-a9be-79e783b85845 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:18.955746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ded42-1bdc-483f-a9be-79e783b85845 None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:18.958388 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ded42-1bdc-483f-a9be-79e783b85845 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:18.960440 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 181/356] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:18 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:18.961279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ded42-1bdc-483f-a9be-79e783b85845 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-e808663f-596f-431b-88c0-cd6b3b843a85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:18.966393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:18.967358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:18.968735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:18.968735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:18.968735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:18.968735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:18.968735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:18.968735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:18.968735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:18.968735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:18.968735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:18.968735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:18.968735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:18.968735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:18.975179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 176/357] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:38:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:18.976073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808663f-596f-431b-88c0-cd6b3b843a85 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:19.081190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808663f-596f-431b-88c0-cd6b3b843a85 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:19.090574 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ded42-1bdc-483f-a9be-79e783b85845 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:19.129286 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f87e75-ef2d-461d-babd-ec656dfa06b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:19.130433 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f87e75-ef2d-461d-babd-ec656dfa06b5 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:19.131023 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f87e75-ef2d-461d-babd-ec656dfa06b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:19.131424 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f87e75-ef2d-461d-babd-ec656dfa06b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:19.142374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820d57e-43ab-4ab9-b9db-66ae96fd7cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:19.143387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d57e-43ab-4ab9-b9db-66ae96fd7cd8 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:19.144041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d57e-43ab-4ab9-b9db-66ae96fd7cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:19.144792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d57e-43ab-4ab9-b9db-66ae96fd7cd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:19.150868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f87e75-ef2d-461d-babd-ec656dfa06b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-63f87e75-ef2d-461d-babd-ec656dfa06b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:19.159673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:19.164252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:19.166009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:19.166009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:19.166009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:19.166009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:19.166009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:19.166009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:19.166009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:19.166009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:19.166009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:19.166009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:19.166009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 21:38:19.166009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:19.166009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 177/358] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:19.166009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f87e75-ef2d-461d-babd-ec656dfa06b5 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:19.252514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820d57e-43ab-4ab9-b9db-66ae96fd7cd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:19.253740 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820d57e-43ab-4ab9-b9db-66ae96fd7cd8 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:19.288655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f87e75-ef2d-461d-babd-ec656dfa06b5 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:19.334683 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10adbbc-79f3-47c7-8b75-804e8eb67a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:19.335303 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10adbbc-79f3-47c7-8b75-804e8eb67a3f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:19.335658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10adbbc-79f3-47c7-8b75-804e8eb67a3f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:19.336110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10adbbc-79f3-47c7-8b75-804e8eb67a3f None admin] PATH_INFO: `/v3/groups/7ea69f2d69684b569f82ad5ec713f402` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:19.337335 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10adbbc-79f3-47c7-8b75-804e8eb67a3f None admin] RBAC: Authorizing `identity:get_group(group_id=7ea69f2d69684b569f82ad5ec713f402)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:19.338981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10adbbc-79f3-47c7-8b75-804e8eb67a3f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:19.342504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 178/359] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:19 2022] GET /identity/v3/groups/7ea69f2d69684b569f82ad5ec713f402 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:19.342504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10adbbc-79f3-47c7-8b75-804e8eb67a3f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:19.375577 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 182/360] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:19.377213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd686a-b87f-411d-a020-1b2c99fa4805 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:19.377504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd686a-b87f-411d-a020-1b2c99fa4805 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:19.377784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd686a-b87f-411d-a020-1b2c99fa4805 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:19.379355 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 183/361] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:19.380261 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd686a-b87f-411d-a020-1b2c99fa4805 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:19.477363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10adbbc-79f3-47c7-8b75-804e8eb67a3f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:19.503983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd686a-b87f-411d-a020-1b2c99fa4805 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:19.513960 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530bc05-5dff-4296-b36a-f7120ac54e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:19.514466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530bc05-5dff-4296-b36a-f7120ac54e1f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:19.514817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530bc05-5dff-4296-b36a-f7120ac54e1f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:19.515103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530bc05-5dff-4296-b36a-f7120ac54e1f None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:19.516093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530bc05-5dff-4296-b36a-f7120ac54e1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:19.516209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530bc05-5dff-4296-b36a-f7120ac54e1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:19.517554 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530bc05-5dff-4296-b36a-f7120ac54e1f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:19.530727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 179/362] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:19.530727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530bc05-5dff-4296-b36a-f7120ac54e1f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:19.545101 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bc94b-727a-471b-b2cd-e8b502393037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:19.546719 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bc94b-727a-471b-b2cd-e8b502393037 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:19.547189 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bc94b-727a-471b-b2cd-e8b502393037 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:19.547672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bc94b-727a-471b-b2cd-e8b502393037 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:19.548597 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bc94b-727a-471b-b2cd-e8b502393037 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:19.549059 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bc94b-727a-471b-b2cd-e8b502393037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:19.550649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bc94b-727a-471b-b2cd-e8b502393037 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:19.559894 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 184/363] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:19.561448 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bc94b-727a-471b-b2cd-e8b502393037 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:19.649549 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530bc05-5dff-4296-b36a-f7120ac54e1f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:19.673550 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bc94b-727a-471b-b2cd-e8b502393037 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:19.698144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d211e-67db-436e-8b42-093de0f719db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:19.699196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d211e-67db-436e-8b42-093de0f719db None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:19.699486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d211e-67db-436e-8b42-093de0f719db None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:19.699769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d211e-67db-436e-8b42-093de0f719db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:19.700674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d211e-67db-436e-8b42-093de0f719db None admin] RBAC: Adding query filter params (group.id=7ea69f2d69684b569f82ad5ec713f402, role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d, scope.project.id=80047f4aaba64866971a8b97d21dee0a) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:19.700998 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d211e-67db-436e-8b42-093de0f719db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:19.702548 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d211e-67db-436e-8b42-093de0f719db None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:19.710844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 180/364] 10.222.0.35 () {56 vars in 1376 bytes} [Fri May 13 21:38:19 2022] GET /identity/v3/role_assignments?group.id=7ea69f2d69684b569f82ad5ec713f402&scope.project.id=80047f4aaba64866971a8b97d21dee0a&role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:19.712562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3369fc5-ff43-4b87-8b7b-2a4fd220d0e3 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:19.712843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3369fc5-ff43-4b87-8b7b-2a4fd220d0e3 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:19.713125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3369fc5-ff43-4b87-8b7b-2a4fd220d0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:19.722525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724f17d-b06f-4226-9654-9d0413dbcaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:19.723430 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724f17d-b06f-4226-9654-9d0413dbcaf4 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:19.723713 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724f17d-b06f-4226-9654-9d0413dbcaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:19.723989 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724f17d-b06f-4226-9654-9d0413dbcaf4 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:19.726338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724f17d-b06f-4226-9654-9d0413dbcaf4 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:19.727672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724f17d-b06f-4226-9654-9d0413dbcaf4 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:19.734852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 185/365] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:19 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:19.736036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724f17d-b06f-4226-9654-9d0413dbcaf4 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:19.741303 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-c3369fc5-ff43-4b87-8b7b-2a4fd220d0e3 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:19.742619 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c3369fc5-ff43-4b87-8b7b-2a4fd220d0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:19.819229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 181/366] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:19.820294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3369fc5-ff43-4b87-8b7b-2a4fd220d0e3 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:19.869800 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724f17d-b06f-4226-9654-9d0413dbcaf4 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:19.929176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34e6e6-6848-492b-9fbd-7901952f8e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:19.930462 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34e6e6-6848-492b-9fbd-7901952f8e85 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:19.930973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34e6e6-6848-492b-9fbd-7901952f8e85 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:19.931460 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34e6e6-6848-492b-9fbd-7901952f8e85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:19.948625 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3369fc5-ff43-4b87-8b7b-2a4fd220d0e3 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:19.989459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c792f4f-88f4-420e-ba0b-07bf0ae5cd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:19.990752 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792f4f-88f4-420e-ba0b-07bf0ae5cd04 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:19.991259 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792f4f-88f4-420e-ba0b-07bf0ae5cd04 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:19.991612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792f4f-88f4-420e-ba0b-07bf0ae5cd04 None admin] PATH_INFO: `/v3/users/0c32fab34ff643a38d9eb1ee975f4c25` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:19.992901 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c792f4f-88f4-420e-ba0b-07bf0ae5cd04 None admin] RBAC: Authorizing `identity:get_user(user_id=0c32fab34ff643a38d9eb1ee975f4c25)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:19.994506 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c792f4f-88f4-420e-ba0b-07bf0ae5cd04 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:19.996562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 182/367] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:38:19 2022] GET /identity/v3/users/0c32fab34ff643a38d9eb1ee975f4c25 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:19.997336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c792f4f-88f4-420e-ba0b-07bf0ae5cd04 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:20.019550 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34e6e6-6848-492b-9fbd-7901952f8e85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:20.021576 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34e6e6-6848-492b-9fbd-7901952f8e85 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:20.114078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c792f4f-88f4-420e-ba0b-07bf0ae5cd04 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:20.154409 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745dc158-4c7b-4b10-acec-6a72ad8e2a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:20.155465 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745dc158-4c7b-4b10-acec-6a72ad8e2a62 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:20.156361 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745dc158-4c7b-4b10-acec-6a72ad8e2a62 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:20.156645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745dc158-4c7b-4b10-acec-6a72ad8e2a62 None admin] PATH_INFO: `/v3/groups/7ea69f2d69684b569f82ad5ec713f402` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:20.157903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745dc158-4c7b-4b10-acec-6a72ad8e2a62 None admin] RBAC: Authorizing `identity:get_group(group_id=7ea69f2d69684b569f82ad5ec713f402)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:20.159382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745dc158-4c7b-4b10-acec-6a72ad8e2a62 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:20.166310 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 183/368] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:19 2022] GET /identity/v3/groups/7ea69f2d69684b569f82ad5ec713f402 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:20.167188 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745dc158-4c7b-4b10-acec-6a72ad8e2a62 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:20.176087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 186/369] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:20.176087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34e6e6-6848-492b-9fbd-7901952f8e85 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:20.297648 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745dc158-4c7b-4b10-acec-6a72ad8e2a62 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:20.301467 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34e6e6-6848-492b-9fbd-7901952f8e85 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:20.334673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9cc44-9338-4a94-82e2-9054aa59ff26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:20.335488 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9cc44-9338-4a94-82e2-9054aa59ff26 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:20.335764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9cc44-9338-4a94-82e2-9054aa59ff26 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:20.336038 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9cc44-9338-4a94-82e2-9054aa59ff26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:20.350727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63fc83f-af89-4c6b-ae40-fe04c0bbe9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:20.350727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fc83f-af89-4c6b-ae40-fe04c0bbe9d7 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:20.350727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fc83f-af89-4c6b-ae40-fe04c0bbe9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:20.350727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fc83f-af89-4c6b-ae40-fe04c0bbe9d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:20.351098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63fc83f-af89-4c6b-ae40-fe04c0bbe9d7 None admin] RBAC: Adding query filter params (role.id=94938baf3b7443bd8e53ce11902c7247, user.id=0c32fab34ff643a38d9eb1ee975f4c25, scope.system=all) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:20.351098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63fc83f-af89-4c6b-ae40-fe04c0bbe9d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:20.351098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63fc83f-af89-4c6b-ae40-fe04c0bbe9d7 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:20.351293 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9cc44-9338-4a94-82e2-9054aa59ff26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-4cd9cc44-9338-4a94-82e2-9054aa59ff26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:20.357910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:20.358950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:20.359753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:20.359753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:20.359753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:20.359753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:20.359753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:20.359753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:20.359753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:20.359753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:20.359753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:20.359753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:20.359753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:38:20.359753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:20.360359 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 184/370] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:38:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:20.360894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9cc44-9338-4a94-82e2-9054aa59ff26 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:20.364591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 187/371] 10.222.0.35 () {56 vars in 1308 bytes} [Fri May 13 21:38:20 2022] GET /identity/v3/role_assignments?user.id=0c32fab34ff643a38d9eb1ee975f4c25&scope.system=all&role.id=94938baf3b7443bd8e53ce11902c7247 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:20.370739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63fc83f-af89-4c6b-ae40-fe04c0bbe9d7 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:20.476359 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9cc44-9338-4a94-82e2-9054aa59ff26 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:20.505890 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63fc83f-af89-4c6b-ae40-fe04c0bbe9d7 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:20.516407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155d6d2-c3d0-41ce-8986-389c15dcae1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:20.517046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155d6d2-c3d0-41ce-8986-389c15dcae1f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:20.517324 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155d6d2-c3d0-41ce-8986-389c15dcae1f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:20.517597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155d6d2-c3d0-41ce-8986-389c15dcae1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:20.518434 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155d6d2-c3d0-41ce-8986-389c15dcae1f None admin] RBAC: Adding query filter params (group.id=7ea69f2d69684b569f82ad5ec713f402, role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d, scope.project.id=a77cd5b44fbe457d82395d5a96360089) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:20.519375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155d6d2-c3d0-41ce-8986-389c15dcae1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:20.520951 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155d6d2-c3d0-41ce-8986-389c15dcae1f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:20.527097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 185/372] 10.222.0.35 () {56 vars in 1376 bytes} [Fri May 13 21:38:20 2022] GET /identity/v3/role_assignments?group.id=7ea69f2d69684b569f82ad5ec713f402&scope.project.id=a77cd5b44fbe457d82395d5a96360089&role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:20.527909 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155d6d2-c3d0-41ce-8986-389c15dcae1f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:20.561559 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb1999-687f-4a1e-8e57-dc0333b81b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:20.564756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb1999-687f-4a1e-8e57-dc0333b81b18 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:20.564756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb1999-687f-4a1e-8e57-dc0333b81b18 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:20.564756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb1999-687f-4a1e-8e57-dc0333b81b18 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:20.565967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb1999-687f-4a1e-8e57-dc0333b81b18 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:20.570110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb1999-687f-4a1e-8e57-dc0333b81b18 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:20.572238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 188/373] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:20 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:20.578282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb1999-687f-4a1e-8e57-dc0333b81b18 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:20.664296 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155d6d2-c3d0-41ce-8986-389c15dcae1f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:20.710419 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175ebd1-3c62-43b4-abbd-7d90400b794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:20.711132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ebd1-3c62-43b4-abbd-7d90400b794d None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:20.711414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ebd1-3c62-43b4-abbd-7d90400b794d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:20.711688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ebd1-3c62-43b4-abbd-7d90400b794d None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:20.712453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175ebd1-3c62-43b4-abbd-7d90400b794d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:20.712757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175ebd1-3c62-43b4-abbd-7d90400b794d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:20.714185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175ebd1-3c62-43b4-abbd-7d90400b794d None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:20.732528 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 186/374] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:20.738292 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175ebd1-3c62-43b4-abbd-7d90400b794d None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:20.783257 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb1999-687f-4a1e-8e57-dc0333b81b18 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:20.832259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5374990-0c94-4cd4-9176-148c1d0d349f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:20.833065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5374990-0c94-4cd4-9176-148c1d0d349f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:20.833341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5374990-0c94-4cd4-9176-148c1d0d349f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:20.833614 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5374990-0c94-4cd4-9176-148c1d0d349f None admin] PATH_INFO: `/v3/groups/7ea69f2d69684b569f82ad5ec713f402` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:20.836047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5374990-0c94-4cd4-9176-148c1d0d349f None admin] RBAC: Authorizing `identity:get_group(group_id=7ea69f2d69684b569f82ad5ec713f402)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:20.837532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5374990-0c94-4cd4-9176-148c1d0d349f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:20.839663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 189/375] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:20 2022] GET /identity/v3/groups/7ea69f2d69684b569f82ad5ec713f402 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:20.845104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5374990-0c94-4cd4-9176-148c1d0d349f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:20.861032 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175ebd1-3c62-43b4-abbd-7d90400b794d None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:20.897105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dcef20-43a5-432e-b8f2-c999e5a86e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:20.898228 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcef20-43a5-432e-b8f2-c999e5a86e65 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:20.898732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcef20-43a5-432e-b8f2-c999e5a86e65 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:20.899240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcef20-43a5-432e-b8f2-c999e5a86e65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:20.970217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dcef20-43a5-432e-b8f2-c999e5a86e65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:20.972102 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dcef20-43a5-432e-b8f2-c999e5a86e65 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:20.978233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5374990-0c94-4cd4-9176-148c1d0d349f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:21.037833 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62222a7a-87a0-4f51-8f67-ad7130e2c984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:21.039354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62222a7a-87a0-4f51-8f67-ad7130e2c984 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:21.040143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62222a7a-87a0-4f51-8f67-ad7130e2c984 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:21.041188 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62222a7a-87a0-4f51-8f67-ad7130e2c984 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:21.042383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62222a7a-87a0-4f51-8f67-ad7130e2c984 None admin] RBAC: Adding query filter params (group.id=7ea69f2d69684b569f82ad5ec713f402, role.id=a29f4364896c46dd84dcee5b969b9898, scope.project.id=a77cd5b44fbe457d82395d5a96360089) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:21.045114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62222a7a-87a0-4f51-8f67-ad7130e2c984 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:21.048384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62222a7a-87a0-4f51-8f67-ad7130e2c984 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:21.062854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 190/376] 10.222.0.35 () {56 vars in 1376 bytes} [Fri May 13 21:38:20 2022] GET /identity/v3/role_assignments?group.id=7ea69f2d69684b569f82ad5ec713f402&scope.project.id=a77cd5b44fbe457d82395d5a96360089&role.id=a29f4364896c46dd84dcee5b969b9898 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:21.090304 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 187/377] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:21.096174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62222a7a-87a0-4f51-8f67-ad7130e2c984 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:21.215053 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62222a7a-87a0-4f51-8f67-ad7130e2c984 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:21.263983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499503a-b697-4539-a586-84030d7783e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:21.264868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499503a-b697-4539-a586-84030d7783e7 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:21.265145 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499503a-b697-4539-a586-84030d7783e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:21.265415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499503a-b697-4539-a586-84030d7783e7 None admin] PATH_INFO: `/v3/projects/44b04a795b974ef6b1191a457380d73c` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:21.268596 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f499503a-b697-4539-a586-84030d7783e7 None admin] RBAC: Authorizing `identity:get_project(project_id=44b04a795b974ef6b1191a457380d73c)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:21.270357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f499503a-b697-4539-a586-84030d7783e7 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:21.272845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 191/378] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:21 2022] GET /identity/v3/projects/44b04a795b974ef6b1191a457380d73c => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:21.278128 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dcef20-43a5-432e-b8f2-c999e5a86e65 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:21.385000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbbdfe-7edc-4e86-b445-1ece73aacc6e None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:21.385603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbbdfe-7edc-4e86-b445-1ece73aacc6e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:21.386099 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbbdfe-7edc-4e86-b445-1ece73aacc6e None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:21.387727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 192/379] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:21.393466 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e2f6b-af7e-45ed-9284-7b879c521ba0 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:21.393757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e2f6b-af7e-45ed-9284-7b879c521ba0 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:21.394038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e2f6b-af7e-45ed-9284-7b879c521ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:21.400504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dcef20-43a5-432e-b8f2-c999e5a86e65 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:21.420747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-527e2f6b-af7e-45ed-9284-7b879c521ba0 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:21.422963 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-527e2f6b-af7e-45ed-9284-7b879c521ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:21.463595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde94ed-3354-475a-a44f-dbbc266f48f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:21.464470 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde94ed-3354-475a-a44f-dbbc266f48f2 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:21.464751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde94ed-3354-475a-a44f-dbbc266f48f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:21.465026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde94ed-3354-475a-a44f-dbbc266f48f2 None admin] PATH_INFO: `/v3/groups/488f6951f8354ed3ab330993f5fbb02e` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:21.475632 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde94ed-3354-475a-a44f-dbbc266f48f2 None admin] RBAC: Authorizing `identity:get_group(group_id=488f6951f8354ed3ab330993f5fbb02e)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:21.477109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde94ed-3354-475a-a44f-dbbc266f48f2 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:21.479889 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 188/380] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:21 2022] GET /identity/v3/groups/488f6951f8354ed3ab330993f5fbb02e => generated 229 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:21.485496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde94ed-3354-475a-a44f-dbbc266f48f2 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:21.545870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 193/381] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:21.589316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e2f6b-af7e-45ed-9284-7b879c521ba0 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:21.651255 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde94ed-3354-475a-a44f-dbbc266f48f2 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:21.694101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e7d23-8eca-429b-b861-03e1e94e8037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:21.694870 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e7d23-8eca-429b-b861-03e1e94e8037 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:21.695054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e7d23-8eca-429b-b861-03e1e94e8037 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:21.695363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e7d23-8eca-429b-b861-03e1e94e8037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:21.696162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e7d23-8eca-429b-b861-03e1e94e8037 None admin] RBAC: Adding query filter params (group.id=488f6951f8354ed3ab330993f5fbb02e, role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=44b04a795b974ef6b1191a457380d73c) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:21.696471 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e7d23-8eca-429b-b861-03e1e94e8037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:21.697916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e7d23-8eca-429b-b861-03e1e94e8037 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:21.711148 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 189/382] 10.222.0.35 () {56 vars in 1376 bytes} [Fri May 13 21:38:21 2022] GET /identity/v3/role_assignments?group.id=488f6951f8354ed3ab330993f5fbb02e&scope.project.id=44b04a795b974ef6b1191a457380d73c&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:21.777968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e2f6b-af7e-45ed-9284-7b879c521ba0 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:21.819782 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71573a3a-1315-45f9-8580-6efad2f6d670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:21.820606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71573a3a-1315-45f9-8580-6efad2f6d670 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:21.820886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71573a3a-1315-45f9-8580-6efad2f6d670 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:21.821159 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71573a3a-1315-45f9-8580-6efad2f6d670 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:21.833103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71573a3a-1315-45f9-8580-6efad2f6d670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-71573a3a-1315-45f9-8580-6efad2f6d670 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:21.841394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:21.842390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:21.843779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:21.843779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:21.843779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:21.843779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:21.843779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:21.843779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:21.843779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:21.843779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:21.843779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:21.843779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:21.843779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:21.843779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:21.843779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 194/383] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:38:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:21.851711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e7d23-8eca-429b-b861-03e1e94e8037 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:21.992114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e7d23-8eca-429b-b861-03e1e94e8037 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:22.036101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3e9fe-9ed9-4f09-9df5-d96b83803a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:22.037149 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3e9fe-9ed9-4f09-9df5-d96b83803a42 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:22.037659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3e9fe-9ed9-4f09-9df5-d96b83803a42 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:22.038145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3e9fe-9ed9-4f09-9df5-d96b83803a42 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:22.039135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3e9fe-9ed9-4f09-9df5-d96b83803a42 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:22.039646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3e9fe-9ed9-4f09-9df5-d96b83803a42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:22.041246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3e9fe-9ed9-4f09-9df5-d96b83803a42 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:22.054660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 190/384] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:22.060420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71573a3a-1315-45f9-8580-6efad2f6d670 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:22.188445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71573a3a-1315-45f9-8580-6efad2f6d670 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:22.229530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509c1f7-d0a6-47de-a112-10d2fe4bbfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:22.230381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509c1f7-d0a6-47de-a112-10d2fe4bbfc2 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:22.230661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509c1f7-d0a6-47de-a112-10d2fe4bbfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:22.230957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509c1f7-d0a6-47de-a112-10d2fe4bbfc2 None admin] PATH_INFO: `/v3/groups/7ea69f2d69684b569f82ad5ec713f402` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:22.232777 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509c1f7-d0a6-47de-a112-10d2fe4bbfc2 None admin] RBAC: Authorizing `identity:get_group(group_id=7ea69f2d69684b569f82ad5ec713f402)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:22.234306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509c1f7-d0a6-47de-a112-10d2fe4bbfc2 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:22.236315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 195/385] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:22 2022] GET /identity/v3/groups/7ea69f2d69684b569f82ad5ec713f402 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:22.242021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3e9fe-9ed9-4f09-9df5-d96b83803a42 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:22.363049 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3e9fe-9ed9-4f09-9df5-d96b83803a42 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:22.400239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004cc020-ab17-4e31-874e-c87f2feb5018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:22.401108 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cc020-ab17-4e31-874e-c87f2feb5018 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:22.401391 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cc020-ab17-4e31-874e-c87f2feb5018 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:22.401666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cc020-ab17-4e31-874e-c87f2feb5018 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:22.402882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004cc020-ab17-4e31-874e-c87f2feb5018 None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:22.404319 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004cc020-ab17-4e31-874e-c87f2feb5018 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:22.406599 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 191/386] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:22 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:22.412234 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509c1f7-d0a6-47de-a112-10d2fe4bbfc2 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:22.531390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509c1f7-d0a6-47de-a112-10d2fe4bbfc2 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:22.575192 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496511ce-1f4a-45ca-8741-ce12b81b30d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:22.576069 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496511ce-1f4a-45ca-8741-ce12b81b30d7 None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:22.576347 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496511ce-1f4a-45ca-8741-ce12b81b30d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:22.576621 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496511ce-1f4a-45ca-8741-ce12b81b30d7 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a/groups/7ea69f2d69684b569f82ad5ec713f402/roles/a29f4364896c46dd84dcee5b969b9898` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:22.584522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496511ce-1f4a-45ca-8741-ce12b81b30d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=80047f4aaba64866971a8b97d21dee0a, group_id=7ea69f2d69684b569f82ad5ec713f402, role_id=a29f4364896c46dd84dcee5b969b9898)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:22.585970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496511ce-1f4a-45ca-8741-ce12b81b30d7 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:22.610940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 196/387] 10.222.0.35 () {58 vars in 1442 bytes} [Fri May 13 21:38:22 2022] PUT /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a/groups/7ea69f2d69684b569f82ad5ec713f402/roles/a29f4364896c46dd84dcee5b969b9898 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:38:22.728167 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e01f29-7f18-4aa6-a128-618024e87a04 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:22.728741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e01f29-7f18-4aa6-a128-618024e87a04 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:22.728741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e01f29-7f18-4aa6-a128-618024e87a04 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:22.730069 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 192/388] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:22.735932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4373d-d82c-41f7-af8b-b578caa53623 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:22.736224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4373d-d82c-41f7-af8b-b578caa53623 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:22.736501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4373d-d82c-41f7-af8b-b578caa53623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:22.755392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-1ec4373d-d82c-41f7-af8b-b578caa53623 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:22.756633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-1ec4373d-d82c-41f7-af8b-b578caa53623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:22.849334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 197/389] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:22.887513 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e01f29-7f18-4aa6-a128-618024e87a04 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:23.005887 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e01f29-7f18-4aa6-a128-618024e87a04 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:23.041731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfef70-24a3-4d06-b0e5-50bd57885b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:23.042538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfef70-24a3-4d06-b0e5-50bd57885b88 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:23.042843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfef70-24a3-4d06-b0e5-50bd57885b88 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:23.043122 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfef70-24a3-4d06-b0e5-50bd57885b88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:23.054237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfef70-24a3-4d06-b0e5-50bd57885b88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-5cbfef70-24a3-4d06-b0e5-50bd57885b88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:23.062480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:23.063363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:23.064173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:23.064173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:23.064173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:23.064173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:23.064173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:23.064173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:23.064173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:23.064173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:23.064173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:23.064173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:23.064173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:23.064173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:23.064173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 193/390] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:23.069684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4373d-d82c-41f7-af8b-b578caa53623 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:23.234878 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4373d-d82c-41f7-af8b-b578caa53623 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:23.276913 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e8169-b3d9-4cdf-9ba9-ed14b38c9cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:23.277504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e8169-b3d9-4cdf-9ba9-ed14b38c9cfe None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:23.277784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e8169-b3d9-4cdf-9ba9-ed14b38c9cfe None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:23.278057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e8169-b3d9-4cdf-9ba9-ed14b38c9cfe None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:23.278822 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e8169-b3d9-4cdf-9ba9-ed14b38c9cfe None admin] RBAC: Adding query filter params (name=member) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:23.279127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e8169-b3d9-4cdf-9ba9-ed14b38c9cfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:23.280535 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e8169-b3d9-4cdf-9ba9-ed14b38c9cfe None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:23.292946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 198/391] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:23.299316 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfef70-24a3-4d06-b0e5-50bd57885b88 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:23.458948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfef70-24a3-4d06-b0e5-50bd57885b88 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:23.506393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c84764-be87-44e8-8fc4-b18d1957d221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:23.507286 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c84764-be87-44e8-8fc4-b18d1957d221 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:23.507633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c84764-be87-44e8-8fc4-b18d1957d221 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:23.507914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c84764-be87-44e8-8fc4-b18d1957d221 None admin] PATH_INFO: `/v3/groups/7ea69f2d69684b569f82ad5ec713f402` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:23.509109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c84764-be87-44e8-8fc4-b18d1957d221 None admin] RBAC: Authorizing `identity:get_group(group_id=7ea69f2d69684b569f82ad5ec713f402)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:23.510566 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c84764-be87-44e8-8fc4-b18d1957d221 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:23.512538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 194/392] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:23 2022] GET /identity/v3/groups/7ea69f2d69684b569f82ad5ec713f402 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:23.518034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e8169-b3d9-4cdf-9ba9-ed14b38c9cfe None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:23.621313 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e8169-b3d9-4cdf-9ba9-ed14b38c9cfe None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:23.648080 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ae1b5-3211-4f82-b0a4-98a085b302b1 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:23.648394 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ae1b5-3211-4f82-b0a4-98a085b302b1 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:23.648641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ae1b5-3211-4f82-b0a4-98a085b302b1 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:23.650932 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 195/393] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:23.656559 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c5d1e-ba47-47da-9003-5566bd016b96 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:23.656848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c5d1e-ba47-47da-9003-5566bd016b96 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:23.657135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c5d1e-ba47-47da-9003-5566bd016b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:23.668818 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc71506-0c8d-42df-8226-fc376cef3c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:23.670000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc71506-0c8d-42df-8226-fc376cef3c7e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:23.670499 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc71506-0c8d-42df-8226-fc376cef3c7e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:23.671588 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc71506-0c8d-42df-8226-fc376cef3c7e None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:23.674237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-f56c5d1e-ba47-47da-9003-5566bd016b96 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:23.675167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc71506-0c8d-42df-8226-fc376cef3c7e None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:23.675469 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f56c5d1e-ba47-47da-9003-5566bd016b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:23.677130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc71506-0c8d-42df-8226-fc376cef3c7e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:23.679394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 199/394] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:23 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:23.685081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc71506-0c8d-42df-8226-fc376cef3c7e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:23.762077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 196/395] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:23.794020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c5d1e-ba47-47da-9003-5566bd016b96 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:23.862240 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc71506-0c8d-42df-8226-fc376cef3c7e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:23.909642 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94777b0-6744-41a5-81bc-c82463af8539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:23.912048 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94777b0-6744-41a5-81bc-c82463af8539 None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:23.913149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94777b0-6744-41a5-81bc-c82463af8539 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:23.914208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94777b0-6744-41a5-81bc-c82463af8539 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a/groups/7ea69f2d69684b569f82ad5ec713f402/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:23.918872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94777b0-6744-41a5-81bc-c82463af8539 None admin] RBAC: Authorizing `identity:create_grant(project_id=80047f4aaba64866971a8b97d21dee0a, group_id=7ea69f2d69684b569f82ad5ec713f402, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:23.923271 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94777b0-6744-41a5-81bc-c82463af8539 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:23.945157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 200/396] 10.222.0.35 () {58 vars in 1442 bytes} [Fri May 13 21:38:23 2022] PUT /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a/groups/7ea69f2d69684b569f82ad5ec713f402/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:38:23.961328 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c5d1e-ba47-47da-9003-5566bd016b96 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:24.006893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7e710-ce34-4de7-85d7-0331d089974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:24.007724 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7e710-ce34-4de7-85d7-0331d089974a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:24.008010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7e710-ce34-4de7-85d7-0331d089974a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:24.008287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7e710-ce34-4de7-85d7-0331d089974a None admin] PATH_INFO: `/v3/roles/member` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:24.033526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7e710-ce34-4de7-85d7-0331d089974a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-69b7e710-ce34-4de7-85d7-0331d089974a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:24.041447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:24.042415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:24.043244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:24.043244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:24.043244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:24.043244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:24.043244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:24.043244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:24.043244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:24.043244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:24.043244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:24.043244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:24.043244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:24.043244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:24.043244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 197/397] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:24.048925 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94777b0-6744-41a5-81bc-c82463af8539 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:24.065708 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08ae30-4726-4717-a74e-2fb5e45a5cfc None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:24.066039 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08ae30-4726-4717-a74e-2fb5e45a5cfc None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:24.066270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08ae30-4726-4717-a74e-2fb5e45a5cfc None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:24.068033 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 198/398] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:24.078475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e56b9-3d8e-4d23-a7b4-ade3e99da596 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:24.078802 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e56b9-3d8e-4d23-a7b4-ade3e99da596 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:24.079063 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e56b9-3d8e-4d23-a7b4-ade3e99da596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:24.108085 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-951e56b9-3d8e-4d23-a7b4-ade3e99da596 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:24.109199 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-951e56b9-3d8e-4d23-a7b4-ade3e99da596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:24.171118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94777b0-6744-41a5-81bc-c82463af8539 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:24.197405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 199/399] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:24.217484 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a188607-3644-42ef-84d9-7e39719a7ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:24.218764 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a188607-3644-42ef-84d9-7e39719a7ec3 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:24.218764 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a188607-3644-42ef-84d9-7e39719a7ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:24.218764 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a188607-3644-42ef-84d9-7e39719a7ec3 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:24.219506 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a188607-3644-42ef-84d9-7e39719a7ec3 None admin] RBAC: Adding query filter params (name=member) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:24.219843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a188607-3644-42ef-84d9-7e39719a7ec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:24.221185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a188607-3644-42ef-84d9-7e39719a7ec3 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:24.231291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 201/400] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:24.233643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e56b9-3d8e-4d23-a7b4-ade3e99da596 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:24.239758 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a188607-3644-42ef-84d9-7e39719a7ec3 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:24.356865 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a188607-3644-42ef-84d9-7e39719a7ec3 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:24.395269 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70826a-882f-4d4c-985d-f274a9785fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:24.396111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70826a-882f-4d4c-985d-f274a9785fcc None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:24.396397 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70826a-882f-4d4c-985d-f274a9785fcc None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:24.396684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70826a-882f-4d4c-985d-f274a9785fcc None admin] PATH_INFO: `/v3/groups/7ea69f2d69684b569f82ad5ec713f402` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:24.398029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70826a-882f-4d4c-985d-f274a9785fcc None admin] RBAC: Authorizing `identity:get_group(group_id=7ea69f2d69684b569f82ad5ec713f402)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:24.399507 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70826a-882f-4d4c-985d-f274a9785fcc None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:24.401455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 202/401] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:24 2022] GET /identity/v3/groups/7ea69f2d69684b569f82ad5ec713f402 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:24.407288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70826a-882f-4d4c-985d-f274a9785fcc None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:24.453533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e56b9-3d8e-4d23-a7b4-ade3e99da596 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:24.517793 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70826a-882f-4d4c-985d-f274a9785fcc None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:24.544002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f086a8-281b-44a4-8277-b9a0ef9197d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:24.545160 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f086a8-281b-44a4-8277-b9a0ef9197d4 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:24.545643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f086a8-281b-44a4-8277-b9a0ef9197d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:24.546639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f086a8-281b-44a4-8277-b9a0ef9197d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:24.553960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d717e83-e7f6-49c3-8d7e-1a7ea36d7f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:24.554954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d717e83-e7f6-49c3-8d7e-1a7ea36d7f80 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:24.555245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d717e83-e7f6-49c3-8d7e-1a7ea36d7f80 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:24.555580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d717e83-e7f6-49c3-8d7e-1a7ea36d7f80 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:24.556966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d717e83-e7f6-49c3-8d7e-1a7ea36d7f80 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:24.558779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d717e83-e7f6-49c3-8d7e-1a7ea36d7f80 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:24.560661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 203/402] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:24 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:24.566826 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d717e83-e7f6-49c3-8d7e-1a7ea36d7f80 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:24.581826 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f086a8-281b-44a4-8277-b9a0ef9197d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-72f086a8-281b-44a4-8277-b9a0ef9197d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:24.597078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:24.598021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:24.598883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:24.598883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:24.598883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:24.598883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:24.598883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:24.598883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:24.598883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:24.598883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:24.598883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:24.598883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:24.598883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:24.598883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:24.611016 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 200/403] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:38:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:24.623796 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f086a8-281b-44a4-8277-b9a0ef9197d4 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:24.681170 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d717e83-e7f6-49c3-8d7e-1a7ea36d7f80 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:24.715132 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9855558-1a7a-4585-991e-9ce0ddb1a0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:24.715939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9855558-1a7a-4585-991e-9ce0ddb1a0df None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:24.716226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9855558-1a7a-4585-991e-9ce0ddb1a0df None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:24.716510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9855558-1a7a-4585-991e-9ce0ddb1a0df None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089/groups/7ea69f2d69684b569f82ad5ec713f402/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:24.718518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9855558-1a7a-4585-991e-9ce0ddb1a0df None admin] RBAC: Authorizing `identity:create_grant(project_id=a77cd5b44fbe457d82395d5a96360089, group_id=7ea69f2d69684b569f82ad5ec713f402, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:24.720000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9855558-1a7a-4585-991e-9ce0ddb1a0df None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:24.740561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 204/404] 10.222.0.35 () {58 vars in 1442 bytes} [Fri May 13 21:38:24 2022] PUT /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089/groups/7ea69f2d69684b569f82ad5ec713f402/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:38:24.801186 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2031d4-6150-46ff-aebc-ec7d3800292b None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:24.801792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2031d4-6150-46ff-aebc-ec7d3800292b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:24.802236 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2031d4-6150-46ff-aebc-ec7d3800292b None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:24.804122 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 205/405] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:24.808956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f086a8-281b-44a4-8277-b9a0ef9197d4 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:24.810239 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e381781-24d1-4b23-8c13-45c273e1919f None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:24.810538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e381781-24d1-4b23-8c13-45c273e1919f None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:24.810868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e381781-24d1-4b23-8c13-45c273e1919f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:24.847809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-9e381781-24d1-4b23-8c13-45c273e1919f None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:24.848702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb3077-1328-4147-bc98-53f637e70d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:24.849014 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9e381781-24d1-4b23-8c13-45c273e1919f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:24.849713 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb3077-1328-4147-bc98-53f637e70d65 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:24.850145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb3077-1328-4147-bc98-53f637e70d65 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:24.850562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb3077-1328-4147-bc98-53f637e70d65 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:24.851792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb3077-1328-4147-bc98-53f637e70d65 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:24.852268 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb3077-1328-4147-bc98-53f637e70d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:24.853867 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb3077-1328-4147-bc98-53f637e70d65 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:24.865047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 201/406] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:24.875159 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb3077-1328-4147-bc98-53f637e70d65 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:24.993583 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 206/407] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:25.016704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e381781-24d1-4b23-8c13-45c273e1919f None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:25.032135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb3077-1328-4147-bc98-53f637e70d65 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:25.073028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a41a6-119b-4107-a7cb-20a776f016a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:25.074277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a41a6-119b-4107-a7cb-20a776f016a7 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:25.074560 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a41a6-119b-4107-a7cb-20a776f016a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:25.074871 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a41a6-119b-4107-a7cb-20a776f016a7 None admin] PATH_INFO: `/v3/groups/7ea69f2d69684b569f82ad5ec713f402` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:25.076415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a41a6-119b-4107-a7cb-20a776f016a7 None admin] RBAC: Authorizing `identity:get_group(group_id=7ea69f2d69684b569f82ad5ec713f402)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:25.078296 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a41a6-119b-4107-a7cb-20a776f016a7 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:25.084906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 202/408] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:24 2022] GET /identity/v3/groups/7ea69f2d69684b569f82ad5ec713f402 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:25.087669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a41a6-119b-4107-a7cb-20a776f016a7 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:25.135530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e381781-24d1-4b23-8c13-45c273e1919f None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:25.169900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35685f44-b4f7-4033-adb1-4797214dd669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:25.170697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35685f44-b4f7-4033-adb1-4797214dd669 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:25.171614 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35685f44-b4f7-4033-adb1-4797214dd669 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:25.171890 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35685f44-b4f7-4033-adb1-4797214dd669 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:25.182392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35685f44-b4f7-4033-adb1-4797214dd669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:25.190541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a41a6-119b-4107-a7cb-20a776f016a7 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-35685f44-b4f7-4033-adb1-4797214dd669 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:25.190660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:25.191503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:25.192267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:25.192267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:25.192267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:25.192267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:25.192267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:25.192267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:25.192267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:25.192267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:25.192267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:25.192267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:25.192267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:38:25.192267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:25.192856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 207/409] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:38:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:25.201826 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35685f44-b4f7-4033-adb1-4797214dd669 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:25.251042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a49227-7dd3-4a0c-b7eb-d177ec706d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:25.251828 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a49227-7dd3-4a0c-b7eb-d177ec706d07 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:25.252114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a49227-7dd3-4a0c-b7eb-d177ec706d07 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:25.252393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a49227-7dd3-4a0c-b7eb-d177ec706d07 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:25.255074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a49227-7dd3-4a0c-b7eb-d177ec706d07 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:25.256521 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a49227-7dd3-4a0c-b7eb-d177ec706d07 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:25.261776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 203/410] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:25 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:25.268160 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a49227-7dd3-4a0c-b7eb-d177ec706d07 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:25.394748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35685f44-b4f7-4033-adb1-4797214dd669 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:25.423204 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a49227-7dd3-4a0c-b7eb-d177ec706d07 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:25.467795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104019ae-25cf-4dae-bf51-d09ed99d4cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:25.468468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104019ae-25cf-4dae-bf51-d09ed99d4cab None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:25.468747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104019ae-25cf-4dae-bf51-d09ed99d4cab None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:25.469021 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104019ae-25cf-4dae-bf51-d09ed99d4cab None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:25.469765 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104019ae-25cf-4dae-bf51-d09ed99d4cab None admin] RBAC: Adding query filter params (name=admin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:25.470069 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104019ae-25cf-4dae-bf51-d09ed99d4cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:25.471502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104019ae-25cf-4dae-bf51-d09ed99d4cab None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:25.483935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7bf06-bafd-42cc-867a-1b6a1db3dea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:25.484870 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7bf06-bafd-42cc-867a-1b6a1db3dea9 None admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:25.485155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7bf06-bafd-42cc-867a-1b6a1db3dea9 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:25.485436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7bf06-bafd-42cc-867a-1b6a1db3dea9 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089/groups/7ea69f2d69684b569f82ad5ec713f402/roles/a29f4364896c46dd84dcee5b969b9898` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:25.487529 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 208/411] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:25.489104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7bf06-bafd-42cc-867a-1b6a1db3dea9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a77cd5b44fbe457d82395d5a96360089, group_id=7ea69f2d69684b569f82ad5ec713f402, role_id=a29f4364896c46dd84dcee5b969b9898)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:25.490848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7bf06-bafd-42cc-867a-1b6a1db3dea9 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:25.492621 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104019ae-25cf-4dae-bf51-d09ed99d4cab None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:25.510764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 204/412] 10.222.0.35 () {58 vars in 1442 bytes} [Fri May 13 21:38:25 2022] PUT /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089/groups/7ea69f2d69684b569f82ad5ec713f402/roles/a29f4364896c46dd84dcee5b969b9898 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:38:25.601090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104019ae-25cf-4dae-bf51-d09ed99d4cab None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:25.606833 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd1f97-d081-4008-80bc-70438fe88bd1 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:25.607474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd1f97-d081-4008-80bc-70438fe88bd1 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:25.607950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd1f97-d081-4008-80bc-70438fe88bd1 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:25.610109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 205/413] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:25.615606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e214f-c6ee-4f17-9bf3-05c35046dbf5 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:25.616168 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e214f-c6ee-4f17-9bf3-05c35046dbf5 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:25.616662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e214f-c6ee-4f17-9bf3-05c35046dbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:25.635151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-fe1e214f-c6ee-4f17-9bf3-05c35046dbf5 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:25.636666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-fe1e214f-c6ee-4f17-9bf3-05c35046dbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:25.647553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ff222-c79c-43e1-9aaf-d4fd287c1240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:25.649835 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ff222-c79c-43e1-9aaf-d4fd287c1240 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:25.651686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ff222-c79c-43e1-9aaf-d4fd287c1240 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:25.652227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ff222-c79c-43e1-9aaf-d4fd287c1240 None admin] PATH_INFO: `/v3/groups/488f6951f8354ed3ab330993f5fbb02e` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:25.656359 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ff222-c79c-43e1-9aaf-d4fd287c1240 None admin] RBAC: Authorizing `identity:get_group(group_id=488f6951f8354ed3ab330993f5fbb02e)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:25.659803 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ff222-c79c-43e1-9aaf-d4fd287c1240 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:25.662780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 209/414] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:25 2022] GET /identity/v3/groups/488f6951f8354ed3ab330993f5fbb02e => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:25.675775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ff222-c79c-43e1-9aaf-d4fd287c1240 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:25.780681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 206/415] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:25.806152 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e214f-c6ee-4f17-9bf3-05c35046dbf5 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:25.822164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ff222-c79c-43e1-9aaf-d4fd287c1240 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:25.871450 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b5335-9291-4011-b2c9-73018f628e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:25.872760 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b5335-9291-4011-b2c9-73018f628e14 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:25.873236 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b5335-9291-4011-b2c9-73018f628e14 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:25.873700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b5335-9291-4011-b2c9-73018f628e14 None admin] PATH_INFO: `/v3/projects/44b04a795b974ef6b1191a457380d73c` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:25.876791 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b5335-9291-4011-b2c9-73018f628e14 None admin] RBAC: Authorizing `identity:get_project(project_id=44b04a795b974ef6b1191a457380d73c)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:25.878718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b5335-9291-4011-b2c9-73018f628e14 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:25.880607 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 210/416] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:25 2022] GET /identity/v3/projects/44b04a795b974ef6b1191a457380d73c => generated 344 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:25.886504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b5335-9291-4011-b2c9-73018f628e14 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:25.979435 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e214f-c6ee-4f17-9bf3-05c35046dbf5 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:26.044226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa25d25-dcda-4219-8a7e-da29ffde5622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:26.045412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa25d25-dcda-4219-8a7e-da29ffde5622 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:26.045918 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa25d25-dcda-4219-8a7e-da29ffde5622 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:26.046381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa25d25-dcda-4219-8a7e-da29ffde5622 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:26.062530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa25d25-dcda-4219-8a7e-da29ffde5622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-7aa25d25-dcda-4219-8a7e-da29ffde5622 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:26.071714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:26.072680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:26.073480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:26.073480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:26.073480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:26.073480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:26.073480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:26.073480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:26.073480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:26.073480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:26.073480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:26.073480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:26.073480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:26.073480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:26.076689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 207/417] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:38:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:26.080645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b5335-9291-4011-b2c9-73018f628e14 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:26.083512 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa25d25-dcda-4219-8a7e-da29ffde5622 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:26.156623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b835787-0b28-442e-b1c0-afd2a0e3bef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:26.157768 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b835787-0b28-442e-b1c0-afd2a0e3bef0 None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:26.159530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b835787-0b28-442e-b1c0-afd2a0e3bef0 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:26.160006 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b835787-0b28-442e-b1c0-afd2a0e3bef0 None admin] PATH_INFO: `/v3/projects/44b04a795b974ef6b1191a457380d73c/groups/488f6951f8354ed3ab330993f5fbb02e/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:26.162980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b835787-0b28-442e-b1c0-afd2a0e3bef0 None admin] RBAC: Authorizing `identity:create_grant(project_id=44b04a795b974ef6b1191a457380d73c, group_id=488f6951f8354ed3ab330993f5fbb02e, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:26.167395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b835787-0b28-442e-b1c0-afd2a0e3bef0 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:26.185507 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 211/418] 10.222.0.35 () {58 vars in 1442 bytes} [Fri May 13 21:38:25 2022] PUT /identity/v3/projects/44b04a795b974ef6b1191a457380d73c/groups/488f6951f8354ed3ab330993f5fbb02e/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:38:26.282045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa25d25-dcda-4219-8a7e-da29ffde5622 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:26.334472 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80e4cf-5d49-48d1-ac36-d179ae0bff45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:26.335096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80e4cf-5d49-48d1-ac36-d179ae0bff45 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:26.335528 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80e4cf-5d49-48d1-ac36-d179ae0bff45 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:26.335814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80e4cf-5d49-48d1-ac36-d179ae0bff45 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:26.336575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80e4cf-5d49-48d1-ac36-d179ae0bff45 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:26.336884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80e4cf-5d49-48d1-ac36-d179ae0bff45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:26.338308 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80e4cf-5d49-48d1-ac36-d179ae0bff45 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:26.350953 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 208/419] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:26.356418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b835787-0b28-442e-b1c0-afd2a0e3bef0 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:26.480339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b835787-0b28-442e-b1c0-afd2a0e3bef0 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:26.533921 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f6300-04fb-430e-8b09-ae53cdebf1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:26.534591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f6300-04fb-430e-8b09-ae53cdebf1ee None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:26.534951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f6300-04fb-430e-8b09-ae53cdebf1ee None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:26.535235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f6300-04fb-430e-8b09-ae53cdebf1ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:26.611173 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f6300-04fb-430e-8b09-ae53cdebf1ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:26.612572 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f6300-04fb-430e-8b09-ae53cdebf1ee None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:26.710729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 212/420] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:26.719794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80e4cf-5d49-48d1-ac36-d179ae0bff45 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:26.870440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80e4cf-5d49-48d1-ac36-d179ae0bff45 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:26.919841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dae2d-18ad-49fb-b1de-f958f6e73f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:26.921767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dae2d-18ad-49fb-b1de-f958f6e73f28 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:26.922179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dae2d-18ad-49fb-b1de-f958f6e73f28 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:26.922610 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dae2d-18ad-49fb-b1de-f958f6e73f28 None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:26.924538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487dae2d-18ad-49fb-b1de-f958f6e73f28 None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:26.927080 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487dae2d-18ad-49fb-b1de-f958f6e73f28 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:26.929601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 209/421] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:26 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:26.934982 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f6300-04fb-430e-8b09-ae53cdebf1ee None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:26.941281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e960a64-d5a3-484a-8ec6-6dd128e4fd2b None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:26.941577 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e960a64-d5a3-484a-8ec6-6dd128e4fd2b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:26.941859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e960a64-d5a3-484a-8ec6-6dd128e4fd2b None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:26.943479 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 210/422] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:26.949331 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d65d01-52be-4608-9a49-83e86e522137 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:26.949624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d65d01-52be-4608-9a49-83e86e522137 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:26.949896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d65d01-52be-4608-9a49-83e86e522137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:26.971090 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-b6d65d01-52be-4608-9a49-83e86e522137 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:26.973931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b6d65d01-52be-4608-9a49-83e86e522137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:27.099795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 211/423] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:27.122682 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d65d01-52be-4608-9a49-83e86e522137 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:27.128461 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f6300-04fb-430e-8b09-ae53cdebf1ee None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:27.192914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25669cfa-230d-4c23-a9b7-7f2f6a13f70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:27.193771 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25669cfa-230d-4c23-a9b7-7f2f6a13f70c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:27.194051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25669cfa-230d-4c23-a9b7-7f2f6a13f70c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:27.194336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25669cfa-230d-4c23-a9b7-7f2f6a13f70c None admin] PATH_INFO: `/v3/groups/7ea69f2d69684b569f82ad5ec713f402` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:27.196227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25669cfa-230d-4c23-a9b7-7f2f6a13f70c None admin] RBAC: Authorizing `identity:get_group(group_id=7ea69f2d69684b569f82ad5ec713f402)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:27.199161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25669cfa-230d-4c23-a9b7-7f2f6a13f70c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:27.203592 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 213/424] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:26 2022] GET /identity/v3/groups/7ea69f2d69684b569f82ad5ec713f402 => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:27.206617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25669cfa-230d-4c23-a9b7-7f2f6a13f70c None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:27.273902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d65d01-52be-4608-9a49-83e86e522137 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:27.312599 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e09326-0853-42e6-9e37-d756e28cdaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:27.313412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e09326-0853-42e6-9e37-d756e28cdaaa None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:27.313693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e09326-0853-42e6-9e37-d756e28cdaaa None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:27.313970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e09326-0853-42e6-9e37-d756e28cdaaa None admin] PATH_INFO: `/v3/roles/member` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:27.325460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e09326-0853-42e6-9e37-d756e28cdaaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-78e09326-0853-42e6-9e37-d756e28cdaaa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:27.333412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:27.334333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:27.335156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:27.335156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:27.335156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:27.335156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:27.335156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:27.335156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:27.335156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:27.335156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:27.335156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:27.335156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:27.335156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:27.335156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:27.335156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 212/425] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:27.340764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e09326-0853-42e6-9e37-d756e28cdaaa None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:27.434377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25669cfa-230d-4c23-a9b7-7f2f6a13f70c None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:27.441285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e09326-0853-42e6-9e37-d756e28cdaaa None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:27.479848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b6590-2048-4eba-a820-bc4acc4d28a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:27.480517 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b6590-2048-4eba-a820-bc4acc4d28a0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:27.480865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b6590-2048-4eba-a820-bc4acc4d28a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:27.481126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b6590-2048-4eba-a820-bc4acc4d28a0 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:27.481920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b6590-2048-4eba-a820-bc4acc4d28a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:27.482226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b6590-2048-4eba-a820-bc4acc4d28a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:27.483665 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b6590-2048-4eba-a820-bc4acc4d28a0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:27.503541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 213/426] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:27.505465 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b6590-2048-4eba-a820-bc4acc4d28a0 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:27.519105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f1dcc-35e4-4dd8-80aa-dd9bba52a9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:27.519769 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f1dcc-35e4-4dd8-80aa-dd9bba52a9e5 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:27.520082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f1dcc-35e4-4dd8-80aa-dd9bba52a9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:27.520322 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f1dcc-35e4-4dd8-80aa-dd9bba52a9e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:27.521158 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f1dcc-35e4-4dd8-80aa-dd9bba52a9e5 None admin] RBAC: Adding query filter params (group.id=7ea69f2d69684b569f82ad5ec713f402, role.id=a29f4364896c46dd84dcee5b969b9898, scope.project.id=80047f4aaba64866971a8b97d21dee0a) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:27.521470 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f1dcc-35e4-4dd8-80aa-dd9bba52a9e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:27.523057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f1dcc-35e4-4dd8-80aa-dd9bba52a9e5 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:27.533938 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 214/427] 10.222.0.35 () {56 vars in 1376 bytes} [Fri May 13 21:38:27 2022] GET /identity/v3/role_assignments?group.id=7ea69f2d69684b569f82ad5ec713f402&scope.project.id=80047f4aaba64866971a8b97d21dee0a&role.id=a29f4364896c46dd84dcee5b969b9898 => generated 611 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:27.556315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d79bc-65ee-4dc8-8c34-4597b63bf023 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:27.556641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d79bc-65ee-4dc8-8c34-4597b63bf023 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:27.556857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d79bc-65ee-4dc8-8c34-4597b63bf023 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:27.562539 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 215/428] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:27.571155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836405ab-bbf5-44dc-8a81-4340009a9866 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:27.571492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836405ab-bbf5-44dc-8a81-4340009a9866 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:27.571775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836405ab-bbf5-44dc-8a81-4340009a9866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:27.607809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-836405ab-bbf5-44dc-8a81-4340009a9866 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:27.609047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-836405ab-bbf5-44dc-8a81-4340009a9866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:27.626817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b6590-2048-4eba-a820-bc4acc4d28a0 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:27.664684 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9749709-0205-49cc-9874-2c90fcc1d2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:27.665362 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9749709-0205-49cc-9874-2c90fcc1d2a8 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:27.665640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9749709-0205-49cc-9874-2c90fcc1d2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:27.665916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9749709-0205-49cc-9874-2c90fcc1d2a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:27.739392 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9749709-0205-49cc-9874-2c90fcc1d2a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:27.740806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9749709-0205-49cc-9874-2c90fcc1d2a8 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:27.753654 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 216/429] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:27.793044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836405ab-bbf5-44dc-8a81-4340009a9866 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:27.847890 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 214/430] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:27.854856 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9749709-0205-49cc-9874-2c90fcc1d2a8 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:27.951655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836405ab-bbf5-44dc-8a81-4340009a9866 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:27.989351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9749709-0205-49cc-9874-2c90fcc1d2a8 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:27.991640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1d43c-0c09-4172-993f-842ba0218375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:27.992885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1d43c-0c09-4172-993f-842ba0218375 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:27.992885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1d43c-0c09-4172-993f-842ba0218375 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:27.993068 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1d43c-0c09-4172-993f-842ba0218375 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:28.008552 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1d43c-0c09-4172-993f-842ba0218375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-dda1d43c-0c09-4172-993f-842ba0218375 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:28.016858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:28.017725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:28.018500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:28.018500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:28.018500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:28.018500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:28.018500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:28.018500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:28.018500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:28.018500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:28.018500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:28.018500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:28.018500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 21:38:28.018500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:28.018500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 217/431] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:28.026383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1d43c-0c09-4172-993f-842ba0218375 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:28.036440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214684b7-a5ec-491d-9112-12438ba9fb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:28.037321 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214684b7-a5ec-491d-9112-12438ba9fb9a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:28.037696 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214684b7-a5ec-491d-9112-12438ba9fb9a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:28.037870 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214684b7-a5ec-491d-9112-12438ba9fb9a None admin] PATH_INFO: `/v3/projects/80047f4aaba64866971a8b97d21dee0a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:28.039096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214684b7-a5ec-491d-9112-12438ba9fb9a None admin] RBAC: Authorizing `identity:get_project(project_id=80047f4aaba64866971a8b97d21dee0a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:28.040735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214684b7-a5ec-491d-9112-12438ba9fb9a None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:28.042522 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 215/432] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:27 2022] GET /identity/v3/projects/80047f4aaba64866971a8b97d21dee0a => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:28.048040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214684b7-a5ec-491d-9112-12438ba9fb9a None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:28.123444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1d43c-0c09-4172-993f-842ba0218375 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:28.163825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e7f94-8d8b-425d-ae4a-ed2dcce75e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:28.164492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e7f94-8d8b-425d-ae4a-ed2dcce75e47 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:28.164771 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e7f94-8d8b-425d-ae4a-ed2dcce75e47 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:28.165038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e7f94-8d8b-425d-ae4a-ed2dcce75e47 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:28.165780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e7f94-8d8b-425d-ae4a-ed2dcce75e47 None admin] RBAC: Adding query filter params (name=member) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:28.166087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e7f94-8d8b-425d-ae4a-ed2dcce75e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:28.167119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214684b7-a5ec-491d-9112-12438ba9fb9a None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:28.167538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e7f94-8d8b-425d-ae4a-ed2dcce75e47 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:28.176234 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 218/433] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:38:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:28.177514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864f608-0142-4c1f-8d75-57ce3639a708 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:28.177854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864f608-0142-4c1f-8d75-57ce3639a708 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:28.178082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864f608-0142-4c1f-8d75-57ce3639a708 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:28.183196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 219/434] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:28.183196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864f608-0142-4c1f-8d75-57ce3639a708 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:28.213601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f9ee7-2fac-4f37-a010-2d7ecfa8a7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:28.215132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f9ee7-2fac-4f37-a010-2d7ecfa8a7e7 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:28.215700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f9ee7-2fac-4f37-a010-2d7ecfa8a7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:28.216194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f9ee7-2fac-4f37-a010-2d7ecfa8a7e7 None admin] PATH_INFO: `/v3/groups/7ea69f2d69684b569f82ad5ec713f402` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:28.217680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f9ee7-2fac-4f37-a010-2d7ecfa8a7e7 None admin] RBAC: Authorizing `identity:get_group(group_id=7ea69f2d69684b569f82ad5ec713f402)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:28.220051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f9ee7-2fac-4f37-a010-2d7ecfa8a7e7 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:28.221860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 216/435] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:28 2022] GET /identity/v3/groups/7ea69f2d69684b569f82ad5ec713f402 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:28.227025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e230b-28cd-4f17-aa9e-513d2595c916 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:28.227025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e230b-28cd-4f17-aa9e-513d2595c916 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:28.227025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e230b-28cd-4f17-aa9e-513d2595c916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:28.242696 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-564e230b-28cd-4f17-aa9e-513d2595c916 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:28.243823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-564e230b-28cd-4f17-aa9e-513d2595c916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:28.290978 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864f608-0142-4c1f-8d75-57ce3639a708 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:28.322725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 217/436] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:28.322725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e230b-28cd-4f17-aa9e-513d2595c916 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:28.334766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37138225-f114-451c-b5e3-d5027c5b53fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:28.335880 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37138225-f114-451c-b5e3-d5027c5b53fb None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:28.336164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37138225-f114-451c-b5e3-d5027c5b53fb None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:28.336493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37138225-f114-451c-b5e3-d5027c5b53fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:28.403066 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37138225-f114-451c-b5e3-d5027c5b53fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:28.404526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37138225-f114-451c-b5e3-d5027c5b53fb None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:28.410361 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e230b-28cd-4f17-aa9e-513d2595c916 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:28.444519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0ec13-6d1b-4bea-aec2-1d74b6d1c316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:28.445163 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0ec13-6d1b-4bea-aec2-1d74b6d1c316 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:28.445446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0ec13-6d1b-4bea-aec2-1d74b6d1c316 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:28.445716 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0ec13-6d1b-4bea-aec2-1d74b6d1c316 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:28.446536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0ec13-6d1b-4bea-aec2-1d74b6d1c316 None admin] RBAC: Adding query filter params (group.id=7ea69f2d69684b569f82ad5ec713f402, role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d, scope.project.id=80047f4aaba64866971a8b97d21dee0a) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:28.446870 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0ec13-6d1b-4bea-aec2-1d74b6d1c316 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:28.448295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0ec13-6d1b-4bea-aec2-1d74b6d1c316 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:28.454351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 218/437] 10.222.0.35 () {56 vars in 1376 bytes} [Fri May 13 21:38:28 2022] GET /identity/v3/role_assignments?group.id=7ea69f2d69684b569f82ad5ec713f402&scope.project.id=80047f4aaba64866971a8b97d21dee0a&role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:28.455145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0ec13-6d1b-4bea-aec2-1d74b6d1c316 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:28.526837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 220/438] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 21:38:28.533184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37138225-f114-451c-b5e3-d5027c5b53fb None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:28.599808 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0ec13-6d1b-4bea-aec2-1d74b6d1c316 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:28.639726 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdf43a-3317-437f-937a-f8bd14bbad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:28.640963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdf43a-3317-437f-937a-f8bd14bbad71 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:28.641246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdf43a-3317-437f-937a-f8bd14bbad71 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:28.641524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdf43a-3317-437f-937a-f8bd14bbad71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:28.654568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdf43a-3317-437f-937a-f8bd14bbad71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-15bdf43a-3317-437f-937a-f8bd14bbad71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:28.663868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:28.664818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:28.665603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:28.665603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:28.665603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:28.665603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:28.665603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:28.665603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:28.665603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:28.665603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:28.665603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:28.665603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:28.665603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:28.665603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:28.667905 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 219/439] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:38:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:28.668869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37138225-f114-451c-b5e3-d5027c5b53fb None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:28.674376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdf43a-3317-437f-937a-f8bd14bbad71 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:28.708538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87e43e-2097-4511-b02e-05bd04192042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:28.709630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e43e-2097-4511-b02e-05bd04192042 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:28.710144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e43e-2097-4511-b02e-05bd04192042 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:28.710608 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e43e-2097-4511-b02e-05bd04192042 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:28.712059 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87e43e-2097-4511-b02e-05bd04192042 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:28.713688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87e43e-2097-4511-b02e-05bd04192042 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:28.717084 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 221/440] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:28 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:28.721556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87e43e-2097-4511-b02e-05bd04192042 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:28.782824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdf43a-3317-437f-937a-f8bd14bbad71 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:28.818413 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca57a62-e0d2-48db-b39f-45100937dabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:28.819102 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca57a62-e0d2-48db-b39f-45100937dabf None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:28.819381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca57a62-e0d2-48db-b39f-45100937dabf None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:28.819669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca57a62-e0d2-48db-b39f-45100937dabf None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:28.820642 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca57a62-e0d2-48db-b39f-45100937dabf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:28.820744 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca57a62-e0d2-48db-b39f-45100937dabf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:28.822032 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca57a62-e0d2-48db-b39f-45100937dabf None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:28.831001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 220/441] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:28.832337 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4198-5441-47a6-b147-0f76478d72b3 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:28.832741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4198-5441-47a6-b147-0f76478d72b3 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:28.832915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4198-5441-47a6-b147-0f76478d72b3 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:28.834263 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 221/442] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:28.838834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4198-5441-47a6-b147-0f76478d72b3 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:28.849490 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87e43e-2097-4511-b02e-05bd04192042 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:28.885856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1aa78-40ba-4010-afdf-4a0f9a703926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:28.886791 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1aa78-40ba-4010-afdf-4a0f9a703926 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:28.887055 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1aa78-40ba-4010-afdf-4a0f9a703926 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:28.887327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1aa78-40ba-4010-afdf-4a0f9a703926 None admin] PATH_INFO: `/v3/groups/7ea69f2d69684b569f82ad5ec713f402` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:28.888524 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1aa78-40ba-4010-afdf-4a0f9a703926 None admin] RBAC: Authorizing `identity:get_group(group_id=7ea69f2d69684b569f82ad5ec713f402)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:28.890008 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1aa78-40ba-4010-afdf-4a0f9a703926 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:28.891649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 222/443] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:28 2022] GET /identity/v3/groups/7ea69f2d69684b569f82ad5ec713f402 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:28.897549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dd3f9-f66b-4b4a-9a55-e7d94c589a19 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:28.897842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dd3f9-f66b-4b4a-9a55-e7d94c589a19 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:28.898118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dd3f9-f66b-4b4a-9a55-e7d94c589a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:28.917033 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a01dd3f9-f66b-4b4a-9a55-e7d94c589a19 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:28.918234 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a01dd3f9-f66b-4b4a-9a55-e7d94c589a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:28.954100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4198-5441-47a6-b147-0f76478d72b3 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:28.990171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 223/444] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:28.991038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dd3f9-f66b-4b4a-9a55-e7d94c589a19 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:28.999408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dce50-2a07-470e-b4df-d197504c157d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:29.000058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dce50-2a07-470e-b4df-d197504c157d None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:29.000337 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dce50-2a07-470e-b4df-d197504c157d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:29.000612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dce50-2a07-470e-b4df-d197504c157d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:29.094211 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01dce50-2a07-470e-b4df-d197504c157d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:29.095695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01dce50-2a07-470e-b4df-d197504c157d None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:29.098817 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dd3f9-f66b-4b4a-9a55-e7d94c589a19 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:29.132216 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58facaf-4351-4cd3-bf7d-6849f4c37cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:29.132864 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58facaf-4351-4cd3-bf7d-6849f4c37cdc None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:29.133148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58facaf-4351-4cd3-bf7d-6849f4c37cdc None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:29.133424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58facaf-4351-4cd3-bf7d-6849f4c37cdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:29.134243 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58facaf-4351-4cd3-bf7d-6849f4c37cdc None admin] RBAC: Adding query filter params (group.id=7ea69f2d69684b569f82ad5ec713f402, role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d, scope.project.id=a77cd5b44fbe457d82395d5a96360089) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:29.134553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58facaf-4351-4cd3-bf7d-6849f4c37cdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:29.136028 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58facaf-4351-4cd3-bf7d-6849f4c37cdc None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:29.142116 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 224/445] 10.222.0.35 () {56 vars in 1376 bytes} [Fri May 13 21:38:28 2022] GET /identity/v3/role_assignments?group.id=7ea69f2d69684b569f82ad5ec713f402&scope.project.id=a77cd5b44fbe457d82395d5a96360089&role.id=fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:29.142975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58facaf-4351-4cd3-bf7d-6849f4c37cdc None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:29.205464 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 222/446] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:29.206894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dce50-2a07-470e-b4df-d197504c157d None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:29.249146 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58facaf-4351-4cd3-bf7d-6849f4c37cdc None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:29.284566 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f101a75-4926-42b0-89ce-46dc16d91167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:29.285367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f101a75-4926-42b0-89ce-46dc16d91167 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:29.285653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f101a75-4926-42b0-89ce-46dc16d91167 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:29.285930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f101a75-4926-42b0-89ce-46dc16d91167 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:29.297217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f101a75-4926-42b0-89ce-46dc16d91167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-4f101a75-4926-42b0-89ce-46dc16d91167 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:29.304991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:29.305931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:29.306729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:29.306729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:29.306729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:29.306729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:29.306729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:29.306729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:29.306729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:29.306729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:29.306729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:29.306729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:29.306729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:38:29.306729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:29.306729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 225/447] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:38:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:29.312772 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f101a75-4926-42b0-89ce-46dc16d91167 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:29.314139 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dce50-2a07-470e-b4df-d197504c157d None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:29.352452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091506d-e449-4288-a910-e924e7317d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:29.353220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091506d-e449-4288-a910-e924e7317d26 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:29.353495 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091506d-e449-4288-a910-e924e7317d26 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:29.353760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091506d-e449-4288-a910-e924e7317d26 None admin] PATH_INFO: `/v3/projects/a77cd5b44fbe457d82395d5a96360089` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:29.354975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091506d-e449-4288-a910-e924e7317d26 None admin] RBAC: Authorizing `identity:get_project(project_id=a77cd5b44fbe457d82395d5a96360089)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:29.356423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091506d-e449-4288-a910-e924e7317d26 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:29.358115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 223/448] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:29 2022] GET /identity/v3/projects/a77cd5b44fbe457d82395d5a96360089 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:29.363541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091506d-e449-4288-a910-e924e7317d26 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:29.424880 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f101a75-4926-42b0-89ce-46dc16d91167 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:29.464017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4cd54-c467-4394-8b31-5377981d8a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:29.464691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4cd54-c467-4394-8b31-5377981d8a37 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:29.464978 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4cd54-c467-4394-8b31-5377981d8a37 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:29.465255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4cd54-c467-4394-8b31-5377981d8a37 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:29.465998 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4cd54-c467-4394-8b31-5377981d8a37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:29.466306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4cd54-c467-4394-8b31-5377981d8a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:29.467753 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4cd54-c467-4394-8b31-5377981d8a37 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:29.474240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091506d-e449-4288-a910-e924e7317d26 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:29.476208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 226/449] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:29.481561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4cd54-c467-4394-8b31-5377981d8a37 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:29.514192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fa3c3-4013-41e1-986b-42cca011ca4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:29.515792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fa3c3-4013-41e1-986b-42cca011ca4a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:29.516391 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fa3c3-4013-41e1-986b-42cca011ca4a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:29.516673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fa3c3-4013-41e1-986b-42cca011ca4a None admin] PATH_INFO: `/v3/groups/7ea69f2d69684b569f82ad5ec713f402` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:29.518034 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fa3c3-4013-41e1-986b-42cca011ca4a None admin] RBAC: Authorizing `identity:get_group(group_id=7ea69f2d69684b569f82ad5ec713f402)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:29.519588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fa3c3-4013-41e1-986b-42cca011ca4a None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:29.521422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 224/450] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:29 2022] GET /identity/v3/groups/7ea69f2d69684b569f82ad5ec713f402 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:29.527357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fa3c3-4013-41e1-986b-42cca011ca4a None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:29.582442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4cd54-c467-4394-8b31-5377981d8a37 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:29.614011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f88e3f-0194-47e4-b79b-cfc406e55298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:29.614580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f88e3f-0194-47e4-b79b-cfc406e55298 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:29.614899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f88e3f-0194-47e4-b79b-cfc406e55298 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:29.615179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f88e3f-0194-47e4-b79b-cfc406e55298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:29.624372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fa3c3-4013-41e1-986b-42cca011ca4a None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:29.656294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502839a-c2f2-4994-b52c-eb8f4277cb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:29.656934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502839a-c2f2-4994-b52c-eb8f4277cb4d None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:29.657218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502839a-c2f2-4994-b52c-eb8f4277cb4d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:29.657496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502839a-c2f2-4994-b52c-eb8f4277cb4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:29.658316 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502839a-c2f2-4994-b52c-eb8f4277cb4d None admin] RBAC: Adding query filter params (group.id=7ea69f2d69684b569f82ad5ec713f402, role.id=a29f4364896c46dd84dcee5b969b9898, scope.project.id=a77cd5b44fbe457d82395d5a96360089) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:29.658628 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502839a-c2f2-4994-b52c-eb8f4277cb4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:29.660083 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502839a-c2f2-4994-b52c-eb8f4277cb4d None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:29.665640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 225/451] 10.222.0.35 () {56 vars in 1376 bytes} [Fri May 13 21:38:29 2022] GET /identity/v3/role_assignments?group.id=7ea69f2d69684b569f82ad5ec713f402&scope.project.id=a77cd5b44fbe457d82395d5a96360089&role.id=a29f4364896c46dd84dcee5b969b9898 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:29.679236 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f88e3f-0194-47e4-b79b-cfc406e55298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:29.680742 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f88e3f-0194-47e4-b79b-cfc406e55298 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:29.781428 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 227/452] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 21:38:29.787202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502839a-c2f2-4994-b52c-eb8f4277cb4d None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:29.886015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502839a-c2f2-4994-b52c-eb8f4277cb4d None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:29.918386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39e8bc-022c-4ad0-a96e-c382e82c8a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:29.919276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39e8bc-022c-4ad0-a96e-c382e82c8a42 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:29.919552 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39e8bc-022c-4ad0-a96e-c382e82c8a42 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:29.919823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39e8bc-022c-4ad0-a96e-c382e82c8a42 None admin] PATH_INFO: `/v3/projects/44b04a795b974ef6b1191a457380d73c` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:29.921535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39e8bc-022c-4ad0-a96e-c382e82c8a42 None admin] RBAC: Authorizing `identity:get_project(project_id=44b04a795b974ef6b1191a457380d73c)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:29.923008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39e8bc-022c-4ad0-a96e-c382e82c8a42 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:29.924703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 226/453] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:38:29 2022] GET /identity/v3/projects/44b04a795b974ef6b1191a457380d73c => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:29.930385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f88e3f-0194-47e4-b79b-cfc406e55298 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:30.028131 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f88e3f-0194-47e4-b79b-cfc406e55298 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:30.060728 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341c940-ce26-4f52-b07b-7ed1fd876966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:30.062140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341c940-ce26-4f52-b07b-7ed1fd876966 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:30.062428 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341c940-ce26-4f52-b07b-7ed1fd876966 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:30.062727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341c940-ce26-4f52-b07b-7ed1fd876966 None admin] PATH_INFO: `/v3/groups/488f6951f8354ed3ab330993f5fbb02e` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:30.063882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341c940-ce26-4f52-b07b-7ed1fd876966 None admin] RBAC: Authorizing `identity:get_group(group_id=488f6951f8354ed3ab330993f5fbb02e)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:30.065331 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341c940-ce26-4f52-b07b-7ed1fd876966 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:30.066970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 228/454] 10.222.0.35 () {56 vars in 1180 bytes} [Fri May 13 21:38:29 2022] GET /identity/v3/groups/488f6951f8354ed3ab330993f5fbb02e => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:30.072666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39e8bc-022c-4ad0-a96e-c382e82c8a42 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:30.165608 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39e8bc-022c-4ad0-a96e-c382e82c8a42 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:30.197017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c0954-e75d-4cfb-a66d-267d0a3d5ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:30.197650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c0954-e75d-4cfb-a66d-267d0a3d5ece None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:30.197928 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c0954-e75d-4cfb-a66d-267d0a3d5ece None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:30.198198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c0954-e75d-4cfb-a66d-267d0a3d5ece None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:30.199052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c0954-e75d-4cfb-a66d-267d0a3d5ece None admin] RBAC: Adding query filter params (group.id=488f6951f8354ed3ab330993f5fbb02e, role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=44b04a795b974ef6b1191a457380d73c) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:30.199356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c0954-e75d-4cfb-a66d-267d0a3d5ece None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:30.200773 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c0954-e75d-4cfb-a66d-267d0a3d5ece None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:30.206305 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 227/455] 10.222.0.35 () {56 vars in 1376 bytes} [Fri May 13 21:38:30 2022] GET /identity/v3/role_assignments?group.id=488f6951f8354ed3ab330993f5fbb02e&scope.project.id=44b04a795b974ef6b1191a457380d73c&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 611 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:33.003581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eff4a-18fb-444d-8e74-eb4fc29d17ee None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.004119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eff4a-18fb-444d-8e74-eb4fc29d17ee None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.004119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eff4a-18fb-444d-8e74-eb4fc29d17ee None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.005978 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 229/456] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:33.012269 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6061b-cafe-4971-9400-82bd08d2fde8 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.012786 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6061b-cafe-4971-9400-82bd08d2fde8 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.012991 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6061b-cafe-4971-9400-82bd08d2fde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.016545 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8adab2-a67c-404d-8fd5-93d11afd0605 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.017533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8adab2-a67c-404d-8fd5-93d11afd0605 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.018553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8adab2-a67c-404d-8fd5-93d11afd0605 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.023383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 230/457] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:33.031524 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ff48e-81bc-453f-9ae0-e43fa696aded None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.032053 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ff48e-81bc-453f-9ae0-e43fa696aded None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.032526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ff48e-81bc-453f-9ae0-e43fa696aded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.043758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-4cc6061b-cafe-4971-9400-82bd08d2fde8 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:33.044959 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4cc6061b-cafe-4971-9400-82bd08d2fde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:33.051713 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-169ff48e-81bc-453f-9ae0-e43fa696aded None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:33.053536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-169ff48e-81bc-453f-9ae0-e43fa696aded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:33.158032 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 228/458] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:33.181257 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420474a0-fe74-4b75-a77a-0d9705a40061 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.181541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420474a0-fe74-4b75-a77a-0d9705a40061 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.181816 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420474a0-fe74-4b75-a77a-0d9705a40061 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.183564 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 229/459] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:33.184101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420474a0-fe74-4b75-a77a-0d9705a40061 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:33.189929 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 231/460] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:33.190297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f0ad9-2da3-49d6-8345-bc7908f2e643 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.190585 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f0ad9-2da3-49d6-8345-bc7908f2e643 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.190900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f0ad9-2da3-49d6-8345-bc7908f2e643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.212052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a02f0ad9-2da3-49d6-8345-bc7908f2e643 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:33.214112 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a02f0ad9-2da3-49d6-8345-bc7908f2e643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:33.297867 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420474a0-fe74-4b75-a77a-0d9705a40061 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:33.298861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 232/461] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:33.307036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f0ad9-2da3-49d6-8345-bc7908f2e643 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:33.345092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b99d7-59fd-4ac3-96ad-340a5fe34ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:33.345903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b99d7-59fd-4ac3-96ad-340a5fe34ea0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.346184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b99d7-59fd-4ac3-96ad-340a5fe34ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.346461 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b99d7-59fd-4ac3-96ad-340a5fe34ea0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.357560 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b99d7-59fd-4ac3-96ad-340a5fe34ea0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:33.359106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b99d7-59fd-4ac3-96ad-340a5fe34ea0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-964b99d7-59fd-4ac3-96ad-340a5fe34ea0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:33.377430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:33.378579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:33.379457 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:33.379457 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:33.379457 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:33.379457 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:33.383678 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 230/462] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:33.384681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b99d7-59fd-4ac3-96ad-340a5fe34ea0 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:33.422754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f0ad9-2da3-49d6-8345-bc7908f2e643 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:33.457862 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd55f4-5a2f-4490-a10e-942305414809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:33.459020 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd55f4-5a2f-4490-a10e-942305414809 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.459521 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd55f4-5a2f-4490-a10e-942305414809 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.459985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd55f4-5a2f-4490-a10e-942305414809 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.470377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd55f4-5a2f-4490-a10e-942305414809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:33.472162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd55f4-5a2f-4490-a10e-942305414809 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:33.485607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b99d7-59fd-4ac3-96ad-340a5fe34ea0 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-9bcd55f4-5a2f-4490-a10e-942305414809 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:33.491510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:33.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:33.493149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:33.493149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:33.493149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:33.493149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:33.495305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 233/463] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:33.496345 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd55f4-5a2f-4490-a10e-942305414809 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:33.521059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5e6ab-4a18-47d0-91cc-628f87a48d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:33.521856 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5e6ab-4a18-47d0-91cc-628f87a48d0b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.522145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5e6ab-4a18-47d0-91cc-628f87a48d0b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.522419 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5e6ab-4a18-47d0-91cc-628f87a48d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.532583 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5e6ab-4a18-47d0-91cc-628f87a48d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:33.534033 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5e6ab-4a18-47d0-91cc-628f87a48d0b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-a7d5e6ab-4a18-47d0-91cc-628f87a48d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:33.555018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:33.555979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:33.557011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:33.557011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:33.557011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:33.557011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:33.558589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 231/464] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:33.559466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5e6ab-4a18-47d0-91cc-628f87a48d0b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:33.603196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd55f4-5a2f-4490-a10e-942305414809 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:33.636183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d3534-8c79-4a8c-abef-698f6c379da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:33.636845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d3534-8c79-4a8c-abef-698f6c379da8 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.637132 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d3534-8c79-4a8c-abef-698f6c379da8 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.637413 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d3534-8c79-4a8c-abef-698f6c379da8 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.638125 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d3534-8c79-4a8c-abef-698f6c379da8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:33.638431 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d3534-8c79-4a8c-abef-698f6c379da8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:33.639884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d3534-8c79-4a8c-abef-698f6c379da8 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:33.655336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 234/465] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:33.655336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92fa5c-84bf-40de-8e59-a56968e404ff None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.655336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92fa5c-84bf-40de-8e59-a56968e404ff None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.655336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92fa5c-84bf-40de-8e59-a56968e404ff None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.655336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 235/466] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:33.655336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92fa5c-84bf-40de-8e59-a56968e404ff None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:33.667193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5e6ab-4a18-47d0-91cc-628f87a48d0b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:33.701392 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8e9f6-ce5d-4eae-9307-3993774ce24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:33.702065 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8e9f6-ce5d-4eae-9307-3993774ce24e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.702349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8e9f6-ce5d-4eae-9307-3993774ce24e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.702649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8e9f6-ce5d-4eae-9307-3993774ce24e None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.703375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8e9f6-ce5d-4eae-9307-3993774ce24e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:33.703681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8e9f6-ce5d-4eae-9307-3993774ce24e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:33.705052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8e9f6-ce5d-4eae-9307-3993774ce24e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:33.716298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 232/467] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:33.717121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8e9f6-ce5d-4eae-9307-3993774ce24e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:33.762418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92fa5c-84bf-40de-8e59-a56968e404ff None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:33.795421 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815ea31-cd22-4bd1-96b5-471b52044fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:33.796611 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815ea31-cd22-4bd1-96b5-471b52044fe3 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.796892 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815ea31-cd22-4bd1-96b5-471b52044fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.797163 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815ea31-cd22-4bd1-96b5-471b52044fe3 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.797871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815ea31-cd22-4bd1-96b5-471b52044fe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:33.798180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815ea31-cd22-4bd1-96b5-471b52044fe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:33.799597 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815ea31-cd22-4bd1-96b5-471b52044fe3 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:33.817735 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 236/468] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:33.817735 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c21eb-e515-4b40-bc9e-f9da874f1f83 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.817987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8e9f6-ce5d-4eae-9307-3993774ce24e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:33.818090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c21eb-e515-4b40-bc9e-f9da874f1f83 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.818090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c21eb-e515-4b40-bc9e-f9da874f1f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.832240 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-b20c21eb-e515-4b40-bc9e-f9da874f1f83 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:33.833445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b20c21eb-e515-4b40-bc9e-f9da874f1f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:33.851001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea67be-23f9-4066-8a14-8a37ab641c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:33.851655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea67be-23f9-4066-8a14-8a37ab641c55 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:33.851931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea67be-23f9-4066-8a14-8a37ab641c55 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:33.852201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea67be-23f9-4066-8a14-8a37ab641c55 None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:33.852897 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea67be-23f9-4066-8a14-8a37ab641c55 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:33.854330 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea67be-23f9-4066-8a14-8a37ab641c55 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:33.885084 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 233/469] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:38:33 2022] POST /identity/v3/users => generated 248 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:38:33.885084 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea67be-23f9-4066-8a14-8a37ab641c55 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:33.930900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 237/470] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:33.933959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c21eb-e515-4b40-bc9e-f9da874f1f83 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:34.024765 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea67be-23f9-4066-8a14-8a37ab641c55 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:34.058506 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c21eb-e515-4b40-bc9e-f9da874f1f83 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:34.066751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c903a81-0d64-4449-b45f-c179b3a05966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:34.066751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903a81-0d64-4449-b45f-c179b3a05966 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:34.066751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903a81-0d64-4449-b45f-c179b3a05966 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:34.066751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903a81-0d64-4449-b45f-c179b3a05966 None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:34.067111 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c903a81-0d64-4449-b45f-c179b3a05966 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:34.067111 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c903a81-0d64-4449-b45f-c179b3a05966 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:34.098807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 234/471] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:38:33 2022] POST /identity/v3/users => generated 246 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:38:34.098986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec75a5-77b0-49f3-a893-752197f341a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:34.099095 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c903a81-0d64-4449-b45f-c179b3a05966 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:34.099174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec75a5-77b0-49f3-a893-752197f341a4 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:34.099174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec75a5-77b0-49f3-a893-752197f341a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:34.099174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec75a5-77b0-49f3-a893-752197f341a4 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:34.099174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec75a5-77b0-49f3-a893-752197f341a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:34.099606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec75a5-77b0-49f3-a893-752197f341a4 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:34.128225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 238/472] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:38:33 2022] POST /identity/v3/users => generated 248 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:38:34.245739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c903a81-0d64-4449-b45f-c179b3a05966 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:34.285070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2db766-a815-40f2-8dfa-0a140f8185bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:34.285895 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2db766-a815-40f2-8dfa-0a140f8185bf None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:34.286177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2db766-a815-40f2-8dfa-0a140f8185bf None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:34.286451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2db766-a815-40f2-8dfa-0a140f8185bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:34.300368 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2db766-a815-40f2-8dfa-0a140f8185bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:34.301956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2db766-a815-40f2-8dfa-0a140f8185bf None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-6a2db766-a815-40f2-8dfa-0a140f8185bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:34.322654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:34.323612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:34.324474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:34.324474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:34.324474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:34.324474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:34.329262 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 235/473] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:34.330807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec75a5-77b0-49f3-a893-752197f341a4 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:34.439187 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec75a5-77b0-49f3-a893-752197f341a4 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:34.474306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bf197-96ac-4147-92cc-23925ac5625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:34.475051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bf197-96ac-4147-92cc-23925ac5625c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:34.475334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bf197-96ac-4147-92cc-23925ac5625c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:34.475612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bf197-96ac-4147-92cc-23925ac5625c None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:34.476337 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bf197-96ac-4147-92cc-23925ac5625c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:34.476643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bf197-96ac-4147-92cc-23925ac5625c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:34.478020 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bf197-96ac-4147-92cc-23925ac5625c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:34.492468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 239/474] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:34.497127 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2db766-a815-40f2-8dfa-0a140f8185bf None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:34.521437 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941185f6-5f48-4bb0-8899-587e7ed5ba8b None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:34.522065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941185f6-5f48-4bb0-8899-587e7ed5ba8b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:34.522604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941185f6-5f48-4bb0-8899-587e7ed5ba8b None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:34.526264 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 240/475] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:34.535156 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636ed43-e08d-4a2f-a73c-88758cad014d None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:34.535789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636ed43-e08d-4a2f-a73c-88758cad014d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:34.536265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636ed43-e08d-4a2f-a73c-88758cad014d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:34.565434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-1636ed43-e08d-4a2f-a73c-88758cad014d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:34.569164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-1636ed43-e08d-4a2f-a73c-88758cad014d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:34.667120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2db766-a815-40f2-8dfa-0a140f8185bf None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:34.705121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 241/476] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:34.716507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb562454-cd7f-47b0-94a0-da73f661eafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:34.717185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb562454-cd7f-47b0-94a0-da73f661eafa None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:34.718010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb562454-cd7f-47b0-94a0-da73f661eafa None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:34.718291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb562454-cd7f-47b0-94a0-da73f661eafa None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:34.719052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb562454-cd7f-47b0-94a0-da73f661eafa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:34.720499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb562454-cd7f-47b0-94a0-da73f661eafa None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:34.729257 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636ed43-e08d-4a2f-a73c-88758cad014d None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:34.744353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 236/477] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:38:34 2022] POST /identity/v3/users => generated 249 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:38:34.874766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636ed43-e08d-4a2f-a73c-88758cad014d None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:34.921910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa0d26-4747-4e56-8461-dbd53fd2fb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:34.922589 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa0d26-4747-4e56-8461-dbd53fd2fb15 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:34.922898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa0d26-4747-4e56-8461-dbd53fd2fb15 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:34.923175 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa0d26-4747-4e56-8461-dbd53fd2fb15 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:34.923931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa0d26-4747-4e56-8461-dbd53fd2fb15 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:34.925315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa0d26-4747-4e56-8461-dbd53fd2fb15 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:34.938312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.sql.core [None req-a1aa0d26-4747-4e56-8461-dbd53fd2fb15 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 13 21:38:34.938312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 13 21:38:34.938312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [parameters: {'id': 'ec50b89521b9417b8330e639665317b8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 13 21:38:34.938312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91430) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-a1aa0d26-4747-4e56-8461-dbd53fd2fb15 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application result = self._query(query) May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application conn.query(q) May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application result.read() May 13 21:38:34.959648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return ref.to_dict() May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 21:38:34.960548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application next(self.gen) May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application yield resource May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application next(self.gen) May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self.session.rollback() May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self.force_reraise() May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise self.value May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application session.commit() May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 13 21:38:34.961458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._prepare_impl() May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self.session.flush() May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._flush(objects) May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application compat.raise_( May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application flush_context.execute() May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rec.execute(self) May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 13 21:38:34.962312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application _emit_insert_statements( May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application c = connection._execute_20( May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ret = self._execute_context( May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 21:38:34.963217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application result = self._query(query) May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application conn.query(q) May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application result.read() May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 21:38:34.964019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [parameters: {'id': 'ec50b89521b9417b8330e639665317b8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:34.964785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 13 21:38:34.965660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:34.966502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 242/478] 10.222.0.35 () {60 vars in 1131 bytes} [Fri May 13 21:38:34 2022] POST /identity/v3/roles => generated 144 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:34.974566 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb562454-cd7f-47b0-94a0-da73f661eafa None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:35.091177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb562454-cd7f-47b0-94a0-da73f661eafa None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:35.126378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a685ab-5dd7-4870-b607-14c0364f84f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:35.127220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a685ab-5dd7-4870-b607-14c0364f84f7 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:35.127501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a685ab-5dd7-4870-b607-14c0364f84f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:35.128074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a685ab-5dd7-4870-b607-14c0364f84f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:35.140053 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a685ab-5dd7-4870-b607-14c0364f84f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-c7a685ab-5dd7-4870-b607-14c0364f84f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:35.148126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:35.148968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:35.149763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:35.149763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:35.149763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:35.149763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:35.149763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:35.149763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:35.149763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:35.149763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:35.149763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:35.149763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:35.149763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 21:38:35.149763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:35.149763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 237/479] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:38:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:35.157778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa0d26-4747-4e56-8461-dbd53fd2fb15 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:35.261002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa0d26-4747-4e56-8461-dbd53fd2fb15 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:35.295638 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91da6fd-e542-4f9f-90c5-011214ebd3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:35.296656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91da6fd-e542-4f9f-90c5-011214ebd3f1 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:35.297113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91da6fd-e542-4f9f-90c5-011214ebd3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:35.297558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91da6fd-e542-4f9f-90c5-011214ebd3f1 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:35.298526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91da6fd-e542-4f9f-90c5-011214ebd3f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:35.299025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91da6fd-e542-4f9f-90c5-011214ebd3f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:35.300596 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91da6fd-e542-4f9f-90c5-011214ebd3f1 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:35.309457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 243/480] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:36.668526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b99cc-7331-4fa5-8723-e4cdeb7b333f None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:36.669576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b99cc-7331-4fa5-8723-e4cdeb7b333f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:36.670060 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b99cc-7331-4fa5-8723-e4cdeb7b333f None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:36.672411 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 238/481] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:36.680309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52930c74-b680-4ed1-a5c6-09e09ed75103 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:36.681168 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52930c74-b680-4ed1-a5c6-09e09ed75103 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:36.681659 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52930c74-b680-4ed1-a5c6-09e09ed75103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:36.703643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-52930c74-b680-4ed1-a5c6-09e09ed75103 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:36.705104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-52930c74-b680-4ed1-a5c6-09e09ed75103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:36.793280 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 244/482] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:36.821589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b99cc-7331-4fa5-8723-e4cdeb7b333f None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:36.974748 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b99cc-7331-4fa5-8723-e4cdeb7b333f None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:36.974942 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a3106-f934-4df0-b19c-1ef865c278c2 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:36.974942 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a3106-f934-4df0-b19c-1ef865c278c2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:36.974942 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a3106-f934-4df0-b19c-1ef865c278c2 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:36.975615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 245/483] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:36.981892 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416945a0-fe5b-43b5-ae59-e91c6740d068 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:36.982178 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416945a0-fe5b-43b5-ae59-e91c6740d068 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:36.982451 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416945a0-fe5b-43b5-ae59-e91c6740d068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:37.017785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f02e3-c4c0-452c-8ed9-34ded4910ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:37.018582 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f02e3-c4c0-452c-8ed9-34ded4910ef0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:37.018887 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f02e3-c4c0-452c-8ed9-34ded4910ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:37.019159 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f02e3-c4c0-452c-8ed9-34ded4910ef0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:37.024751 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-416945a0-fe5b-43b5-ae59-e91c6740d068 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:37.026798 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-416945a0-fe5b-43b5-ae59-e91c6740d068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:37.047422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f02e3-c4c0-452c-8ed9-34ded4910ef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-cc4f02e3-c4c0-452c-8ed9-34ded4910ef0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:37.055536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:37.056446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:37.057402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:37.057402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:37.057402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:37.057402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:37.057402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:37.057402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:37.057402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:37.057402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:37.057402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:37.057402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:37.057402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:37.057402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:37.057402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 239/484] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:37.058249 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07896c3-1eca-4a4a-9656-a77c5f348aaf None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:37.058543 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07896c3-1eca-4a4a-9656-a77c5f348aaf None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:37.062931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07896c3-1eca-4a4a-9656-a77c5f348aaf None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:37.065150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 240/485] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:37.066393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07896c3-1eca-4a4a-9656-a77c5f348aaf None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:37.115053 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 246/486] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:37.116667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f0b8b-8a36-46b6-9f39-c39eafd4b937 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:37.117126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f0b8b-8a36-46b6-9f39-c39eafd4b937 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:37.117545 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f0b8b-8a36-46b6-9f39-c39eafd4b937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:37.133845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-442f0b8b-8a36-46b6-9f39-c39eafd4b937 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:37.135256 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-442f0b8b-8a36-46b6-9f39-c39eafd4b937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:37.206652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07896c3-1eca-4a4a-9656-a77c5f348aaf None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:37.218730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 247/487] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:37.218730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f0b8b-8a36-46b6-9f39-c39eafd4b937 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:37.270017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132f0ab-cf18-4354-91db-60245431f7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:37.271044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132f0ab-cf18-4354-91db-60245431f7b4 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:37.271631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132f0ab-cf18-4354-91db-60245431f7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:37.272057 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132f0ab-cf18-4354-91db-60245431f7b4 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:37.273611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132f0ab-cf18-4354-91db-60245431f7b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:37.274069 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132f0ab-cf18-4354-91db-60245431f7b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:37.276094 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132f0ab-cf18-4354-91db-60245431f7b4 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:37.288066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 241/488] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:37.289341 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132f0ab-cf18-4354-91db-60245431f7b4 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:37.334465 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f0b8b-8a36-46b6-9f39-c39eafd4b937 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:37.385853 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a905b9-ae35-47a4-8630-9e24360a6066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:37.386979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a905b9-ae35-47a4-8630-9e24360a6066 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:37.387434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a905b9-ae35-47a4-8630-9e24360a6066 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:37.387923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a905b9-ae35-47a4-8630-9e24360a6066 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:37.403017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a905b9-ae35-47a4-8630-9e24360a6066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-62a905b9-ae35-47a4-8630-9e24360a6066 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:37.414932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:37.416016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:37.416898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:37.416898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:37.416898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:37.416898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:37.416898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:37.416898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:37.416898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:37.416898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:37.416898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:37.416898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:37.416898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:37.416898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:37.419634 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 248/489] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:37.421276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a905b9-ae35-47a4-8630-9e24360a6066 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:37.434765 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132f0ab-cf18-4354-91db-60245431f7b4 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:37.468778 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eb427-b5be-424a-b988-fa752343bc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:37.469571 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eb427-b5be-424a-b988-fa752343bc4a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:37.469858 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eb427-b5be-424a-b988-fa752343bc4a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:37.470133 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eb427-b5be-424a-b988-fa752343bc4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:37.490841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1eb427-b5be-424a-b988-fa752343bc4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-9e1eb427-b5be-424a-b988-fa752343bc4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:37.498456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:37.499416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:37.500448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:37.500448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:37.500448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:37.500448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:37.500448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:37.500448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:37.500448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:37.500448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:37.500448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:37.500448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:37.500448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:37.500448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:37.500448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 242/490] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:37.501282 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eb427-b5be-424a-b988-fa752343bc4a None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:37.552417 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a905b9-ae35-47a4-8630-9e24360a6066 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:37.590325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4654359-7015-44f0-9c44-91f27a040f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:37.591394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4654359-7015-44f0-9c44-91f27a040f05 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:37.591748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4654359-7015-44f0-9c44-91f27a040f05 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:37.592387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4654359-7015-44f0-9c44-91f27a040f05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:37.605295 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4654359-7015-44f0-9c44-91f27a040f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:37.606816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4654359-7015-44f0-9c44-91f27a040f05 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:37.625645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eb427-b5be-424a-b988-fa752343bc4a None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-a4654359-7015-44f0-9c44-91f27a040f05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:37.626079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:37.627351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:37.629391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:37.629391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:37.629391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:37.629391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:37.632142 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 249/491] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:37.632916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4654359-7015-44f0-9c44-91f27a040f05 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:37.663872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c05242-911b-4738-b2e3-92a1f5f6eb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:37.664854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c05242-911b-4738-b2e3-92a1f5f6eb2e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:37.665137 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c05242-911b-4738-b2e3-92a1f5f6eb2e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:37.665408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c05242-911b-4738-b2e3-92a1f5f6eb2e None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:37.666365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c05242-911b-4738-b2e3-92a1f5f6eb2e None admin] RBAC: Adding query filter params (name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:37.666672 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c05242-911b-4738-b2e3-92a1f5f6eb2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:37.668940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c05242-911b-4738-b2e3-92a1f5f6eb2e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:37.677614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 243/492] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:37.679246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ceeda-2e3e-43e6-83c0-c2b9a32c0f16 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:37.679608 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ceeda-2e3e-43e6-83c0-c2b9a32c0f16 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:37.679958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ceeda-2e3e-43e6-83c0-c2b9a32c0f16 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:37.681423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 244/493] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:37.683751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ceeda-2e3e-43e6-83c0-c2b9a32c0f16 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:37.734507 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4654359-7015-44f0-9c44-91f27a040f05 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:37.768603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc025f4-e9a6-472d-ae2d-6cadc43afdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:37.769486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc025f4-e9a6-472d-ae2d-6cadc43afdd7 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:37.769594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc025f4-e9a6-472d-ae2d-6cadc43afdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:37.769815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc025f4-e9a6-472d-ae2d-6cadc43afdd7 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:37.770638 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc025f4-e9a6-472d-ae2d-6cadc43afdd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:37.770899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc025f4-e9a6-472d-ae2d-6cadc43afdd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:37.772285 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc025f4-e9a6-472d-ae2d-6cadc43afdd7 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:37.780228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 250/494] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:37.781023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc025f4-e9a6-472d-ae2d-6cadc43afdd7 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:37.792438 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ceeda-2e3e-43e6-83c0-c2b9a32c0f16 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:37.826179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73524d7-afd1-4ce2-a7a8-b0b778f3bf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:37.827424 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73524d7-afd1-4ce2-a7a8-b0b778f3bf99 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:37.827710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73524d7-afd1-4ce2-a7a8-b0b778f3bf99 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:37.827992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73524d7-afd1-4ce2-a7a8-b0b778f3bf99 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:37.828716 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73524d7-afd1-4ce2-a7a8-b0b778f3bf99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:37.829021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73524d7-afd1-4ce2-a7a8-b0b778f3bf99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:37.830409 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73524d7-afd1-4ce2-a7a8-b0b778f3bf99 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:37.844111 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 245/495] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:37.845237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256aaff9-3814-4a23-9697-ccc9619380ed None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:37.845521 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256aaff9-3814-4a23-9697-ccc9619380ed None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:37.845796 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256aaff9-3814-4a23-9697-ccc9619380ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:37.865104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-256aaff9-3814-4a23-9697-ccc9619380ed None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:37.867268 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-256aaff9-3814-4a23-9697-ccc9619380ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:37.893775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc025f4-e9a6-472d-ae2d-6cadc43afdd7 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:37.946737 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 246/496] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:37.946737 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256aaff9-3814-4a23-9697-ccc9619380ed None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:37.948751 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388cdde-bb11-4141-9e51-a400600416be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:37.949550 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388cdde-bb11-4141-9e51-a400600416be None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:37.949832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388cdde-bb11-4141-9e51-a400600416be None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:37.950114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388cdde-bb11-4141-9e51-a400600416be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:37.961481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388cdde-bb11-4141-9e51-a400600416be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:37.962982 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388cdde-bb11-4141-9e51-a400600416be None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-7388cdde-bb11-4141-9e51-a400600416be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:37.980660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:37.981543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:37.982382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:37.982382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:37.982382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:37.982382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:37.982382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 251/497] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:37.982647 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388cdde-bb11-4141-9e51-a400600416be None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:38.050184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256aaff9-3814-4a23-9697-ccc9619380ed None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:38.084373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddd0cc-078b-478a-b65d-56be67796865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:38.085544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddd0cc-078b-478a-b65d-56be67796865 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:38.085974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddd0cc-078b-478a-b65d-56be67796865 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:38.086375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddd0cc-078b-478a-b65d-56be67796865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:38.088107 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388cdde-bb11-4141-9e51-a400600416be None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:38.098260 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddd0cc-078b-478a-b65d-56be67796865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:38.099751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddd0cc-078b-478a-b65d-56be67796865 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-e5ddd0cc-078b-478a-b65d-56be67796865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:38.117482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:38.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:38.119246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:38.119246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:38.119246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:38.119246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:38.119246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 247/498] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:38.119505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddd0cc-078b-478a-b65d-56be67796865 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:38.128192 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8af3b-e639-44d9-ab3b-765b6cc9d461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:38.128995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8af3b-e639-44d9-ab3b-765b6cc9d461 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:38.129278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8af3b-e639-44d9-ab3b-765b6cc9d461 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:38.129549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8af3b-e639-44d9-ab3b-765b6cc9d461 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:38.137213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8af3b-e639-44d9-ab3b-765b6cc9d461 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:38.138672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8af3b-e639-44d9-ab3b-765b6cc9d461 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-b4e8af3b-e639-44d9-ab3b-765b6cc9d461 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:38.156195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:38.157129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:38.158034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:38.158034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:38.158034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:38.158034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:38.158034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:38.158034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:38.158034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:38.158034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:38.158034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 13 21:38:38.158034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:38.158034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 252/499] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:38.158598 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8af3b-e639-44d9-ab3b-765b6cc9d461 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:38.230188 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddd0cc-078b-478a-b65d-56be67796865 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:38.263683 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc075948-b3b9-41e2-a594-66c0b24cb8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:38.264993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc075948-b3b9-41e2-a594-66c0b24cb8d1 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:38.265098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8af3b-e639-44d9-ab3b-765b6cc9d461 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:38.265263 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc075948-b3b9-41e2-a594-66c0b24cb8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:38.265531 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc075948-b3b9-41e2-a594-66c0b24cb8d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:38.276245 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc075948-b3b9-41e2-a594-66c0b24cb8d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-dc075948-b3b9-41e2-a594-66c0b24cb8d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:38.284432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:38.285504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:38.286385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:38.286385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:38.286385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:38.286385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:38.286385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:38.286385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:38.286385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:38.286385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:38.286385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:38.286385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:38.286385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:38.286385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:38.286385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 248/500] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:38.287633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc075948-b3b9-41e2-a594-66c0b24cb8d1 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:38.303839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a4a63-f6a8-4aae-94b7-71a4e47b1dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:38.304508 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a4a63-f6a8-4aae-94b7-71a4e47b1dbe None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:38.304793 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a4a63-f6a8-4aae-94b7-71a4e47b1dbe None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:38.305065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a4a63-f6a8-4aae-94b7-71a4e47b1dbe None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:38.305789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a4a63-f6a8-4aae-94b7-71a4e47b1dbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:38.306096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a4a63-f6a8-4aae-94b7-71a4e47b1dbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:38.307528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a4a63-f6a8-4aae-94b7-71a4e47b1dbe None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:38.324103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 253/501] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:38.324103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a4a63-f6a8-4aae-94b7-71a4e47b1dbe None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:38.386988 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc075948-b3b9-41e2-a594-66c0b24cb8d1 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:38.416475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a4a63-f6a8-4aae-94b7-71a4e47b1dbe None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:38.418985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca956a05-7fad-43b9-95c9-c9ca8d8427e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:38.419645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca956a05-7fad-43b9-95c9-c9ca8d8427e8 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:38.419924 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca956a05-7fad-43b9-95c9-c9ca8d8427e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:38.420196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca956a05-7fad-43b9-95c9-c9ca8d8427e8 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:38.421120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca956a05-7fad-43b9-95c9-c9ca8d8427e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:38.421424 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca956a05-7fad-43b9-95c9-c9ca8d8427e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:38.422836 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca956a05-7fad-43b9-95c9-c9ca8d8427e8 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:38.434840 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 249/502] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:38.435678 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca956a05-7fad-43b9-95c9-c9ca8d8427e8 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:38.450308 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e95d9-31d3-47c1-afed-468bbabe8984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:38.451062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e95d9-31d3-47c1-afed-468bbabe8984 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:38.451344 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e95d9-31d3-47c1-afed-468bbabe8984 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:38.451721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e95d9-31d3-47c1-afed-468bbabe8984 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:38.453083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e95d9-31d3-47c1-afed-468bbabe8984 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:38.453392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e95d9-31d3-47c1-afed-468bbabe8984 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:38.454900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e95d9-31d3-47c1-afed-468bbabe8984 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:38.490688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 254/503] 10.222.0.35 () {56 vars in 1137 bytes} [Fri May 13 21:38:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:38.491560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e95d9-31d3-47c1-afed-468bbabe8984 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:38.534045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca956a05-7fad-43b9-95c9-c9ca8d8427e8 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:38.566399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68b774-5a6f-47d6-bab8-ee013d9324bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:38.567103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68b774-5a6f-47d6-bab8-ee013d9324bf None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:38.567387 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68b774-5a6f-47d6-bab8-ee013d9324bf None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:38.567659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68b774-5a6f-47d6-bab8-ee013d9324bf None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:38.568398 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68b774-5a6f-47d6-bab8-ee013d9324bf None admin] RBAC: Adding query filter params (name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:38.568700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68b774-5a6f-47d6-bab8-ee013d9324bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:38.570068 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68b774-5a6f-47d6-bab8-ee013d9324bf None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:38.578578 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 250/504] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:38.579949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be710e1e-8967-47a3-8c9a-31e44e8cb037 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:38.580248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be710e1e-8967-47a3-8c9a-31e44e8cb037 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:38.580523 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be710e1e-8967-47a3-8c9a-31e44e8cb037 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:38.581905 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 251/505] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:38.582509 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be710e1e-8967-47a3-8c9a-31e44e8cb037 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:38.598384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e95d9-31d3-47c1-afed-468bbabe8984 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:38.631221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0ddec-4596-48b0-b589-ee026b6f8eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:38.631999 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0ddec-4596-48b0-b589-ee026b6f8eb0 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:38.632279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0ddec-4596-48b0-b589-ee026b6f8eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:38.632575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0ddec-4596-48b0-b589-ee026b6f8eb0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:38.640824 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0ddec-4596-48b0-b589-ee026b6f8eb0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:38.642214 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0ddec-4596-48b0-b589-ee026b6f8eb0 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-51a0ddec-4596-48b0-b589-ee026b6f8eb0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:38.656481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:38.657600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:38.658519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:38.658519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:38.658519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:38.658519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:38.658519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:38.658519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:38.658519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:38.658519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:38.658519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 13 21:38:38.658519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:38.658519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 255/506] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:38.658519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0ddec-4596-48b0-b589-ee026b6f8eb0 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:38.687798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be710e1e-8967-47a3-8c9a-31e44e8cb037 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:38.720046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56aa1-6117-4b90-a121-3f24d37a7a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:38.720830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56aa1-6117-4b90-a121-3f24d37a7a0a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:38.721111 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56aa1-6117-4b90-a121-3f24d37a7a0a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:38.721388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56aa1-6117-4b90-a121-3f24d37a7a0a None admin] PATH_INFO: `/v3/users/nova` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:38.728670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed56aa1-6117-4b90-a121-3f24d37a7a0a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:38.730103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed56aa1-6117-4b90-a121-3f24d37a7a0a None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-9ed56aa1-6117-4b90-a121-3f24d37a7a0a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:38.745700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:38.746859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:38.747738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:38.747738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:38.747738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:38.747738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:38.747738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:38.747738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:38.747738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:38.747738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:38.747738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 21:38:38.747738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:38.747738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 252/507] 10.222.0.35 () {56 vars in 1093 bytes} [Fri May 13 21:38:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:38.747738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56aa1-6117-4b90-a121-3f24d37a7a0a None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:38.751919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0ddec-4596-48b0-b589-ee026b6f8eb0 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:38.783325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40fda8-f787-4110-9058-f9b7408f669f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:38.783956 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40fda8-f787-4110-9058-f9b7408f669f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:38.784230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40fda8-f787-4110-9058-f9b7408f669f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:38.784504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40fda8-f787-4110-9058-f9b7408f669f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:38.842310 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56aa1-6117-4b90-a121-3f24d37a7a0a None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:38.848784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40fda8-f787-4110-9058-f9b7408f669f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:38.850237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40fda8-f787-4110-9058-f9b7408f669f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:38.874974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b7583-c565-4803-86ad-738a93516753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:38.875962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b7583-c565-4803-86ad-738a93516753 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:38.876248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b7583-c565-4803-86ad-738a93516753 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:38.876523 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b7583-c565-4803-86ad-738a93516753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:38.887808 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b7583-c565-4803-86ad-738a93516753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:38.889345 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b7583-c565-4803-86ad-738a93516753 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-ae9b7583-c565-4803-86ad-738a93516753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:38.905516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:38.906533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:38.907641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:38.907641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:38.907641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:38.907641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:38.907641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 253/508] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:38.908073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6a43b-c5ec-4ace-a1bb-03de84fde951 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:38.908354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6a43b-c5ec-4ace-a1bb-03de84fde951 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:38.908659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6a43b-c5ec-4ace-a1bb-03de84fde951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:38.924461 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-8da6a43b-c5ec-4ace-a1bb-03de84fde951 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:38.925644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8da6a43b-c5ec-4ace-a1bb-03de84fde951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:38.948553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 256/509] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:38.949373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40fda8-f787-4110-9058-f9b7408f669f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:38.988422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 254/510] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:38.989267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6a43b-c5ec-4ace-a1bb-03de84fde951 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:39.050799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40fda8-f787-4110-9058-f9b7408f669f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:39.082538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cad66-aa78-421f-9c5d-1f75a2974df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:39.083220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cad66-aa78-421f-9c5d-1f75a2974df4 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:39.083508 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cad66-aa78-421f-9c5d-1f75a2974df4 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:39.083784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cad66-aa78-421f-9c5d-1f75a2974df4 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:39.084059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6a43b-c5ec-4ace-a1bb-03de84fde951 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:39.084611 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08cad66-aa78-421f-9c5d-1f75a2974df4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:39.084923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08cad66-aa78-421f-9c5d-1f75a2974df4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:39.086374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08cad66-aa78-421f-9c5d-1f75a2974df4 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:39.103194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 257/511] 10.222.0.35 () {56 vars in 1137 bytes} [Fri May 13 21:38:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:39.103792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cad66-aa78-421f-9c5d-1f75a2974df4 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:39.117492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff16343-58fc-4200-bd04-ff011684c934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:39.118234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff16343-58fc-4200-bd04-ff011684c934 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:39.118586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff16343-58fc-4200-bd04-ff011684c934 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:39.118897 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff16343-58fc-4200-bd04-ff011684c934 None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:39.119731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff16343-58fc-4200-bd04-ff011684c934 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:39.120037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff16343-58fc-4200-bd04-ff011684c934 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:39.121459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff16343-58fc-4200-bd04-ff011684c934 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:39.158147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 255/512] 10.222.0.35 () {56 vars in 1133 bytes} [Fri May 13 21:38:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:39.158772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff16343-58fc-4200-bd04-ff011684c934 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:39.198001 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cad66-aa78-421f-9c5d-1f75a2974df4 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:39.229688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23076ede-1867-4e61-a2fb-f558148d49ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:39.230333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23076ede-1867-4e61-a2fb-f558148d49ae None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:39.230614 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23076ede-1867-4e61-a2fb-f558148d49ae None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:39.230919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23076ede-1867-4e61-a2fb-f558148d49ae None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:39.231633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23076ede-1867-4e61-a2fb-f558148d49ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:39.231937 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23076ede-1867-4e61-a2fb-f558148d49ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:39.233354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23076ede-1867-4e61-a2fb-f558148d49ae None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:39.244211 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 258/513] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:39.244991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23076ede-1867-4e61-a2fb-f558148d49ae None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:39.256298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff16343-58fc-4200-bd04-ff011684c934 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:39.288349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d3cd2-653b-4c9d-9ded-fd0dc729efea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:39.289130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d3cd2-653b-4c9d-9ded-fd0dc729efea None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:39.289413 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d3cd2-653b-4c9d-9ded-fd0dc729efea None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:39.289684 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d3cd2-653b-4c9d-9ded-fd0dc729efea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:39.299931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d3cd2-653b-4c9d-9ded-fd0dc729efea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:39.301423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d3cd2-653b-4c9d-9ded-fd0dc729efea None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-4f3d3cd2-653b-4c9d-9ded-fd0dc729efea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:39.318947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:39.319798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:39.320630 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:39.320630 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:39.320630 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:39.320630 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:39.320630 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 256/514] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:39.321053 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d3cd2-653b-4c9d-9ded-fd0dc729efea None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:39.344822 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23076ede-1867-4e61-a2fb-f558148d49ae None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:39.376089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa5787-e610-402e-a5e5-0758def2e0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:39.376862 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa5787-e610-402e-a5e5-0758def2e0c4 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:39.377143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa5787-e610-402e-a5e5-0758def2e0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:39.377419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa5787-e610-402e-a5e5-0758def2e0c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:39.387598 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa5787-e610-402e-a5e5-0758def2e0c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:39.389040 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa5787-e610-402e-a5e5-0758def2e0c4 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-5cfa5787-e610-402e-a5e5-0758def2e0c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:39.405804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:39.406662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:39.407534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:39.407534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:39.407534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:39.407534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:39.407534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 259/515] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:39.408118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa5787-e610-402e-a5e5-0758def2e0c4 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:39.418478 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d3cd2-653b-4c9d-9ded-fd0dc729efea None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:39.452507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651143e-d14c-4425-a58b-cdb4111d5f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:39.453146 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651143e-d14c-4425-a58b-cdb4111d5f80 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:39.453428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651143e-d14c-4425-a58b-cdb4111d5f80 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:39.453701 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651143e-d14c-4425-a58b-cdb4111d5f80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:39.505840 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa5787-e610-402e-a5e5-0758def2e0c4 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:39.518096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651143e-d14c-4425-a58b-cdb4111d5f80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:39.519597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651143e-d14c-4425-a58b-cdb4111d5f80 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:39.537328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c6e94-3b7e-44e3-b6a0-32f794ac1c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:39.537953 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c6e94-3b7e-44e3-b6a0-32f794ac1c44 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:39.538233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c6e94-3b7e-44e3-b6a0-32f794ac1c44 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:39.538505 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c6e94-3b7e-44e3-b6a0-32f794ac1c44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:39.605358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c6e94-3b7e-44e3-b6a0-32f794ac1c44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:39.606864 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c6e94-3b7e-44e3-b6a0-32f794ac1c44 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:39.615125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 257/516] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:39.615974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651143e-d14c-4425-a58b-cdb4111d5f80 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:39.703155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 260/517] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:39.704339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c6e94-3b7e-44e3-b6a0-32f794ac1c44 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:39.709319 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651143e-d14c-4425-a58b-cdb4111d5f80 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:39.741426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550a31c-5cca-4527-a670-9cb87476569f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:39.742184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550a31c-5cca-4527-a670-9cb87476569f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:39.742463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550a31c-5cca-4527-a670-9cb87476569f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:39.742763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550a31c-5cca-4527-a670-9cb87476569f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:39.749523 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550a31c-5cca-4527-a670-9cb87476569f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:39.751030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550a31c-5cca-4527-a670-9cb87476569f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-e550a31c-5cca-4527-a670-9cb87476569f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:39.765830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:39.766850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:39.767754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:39.767754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:39.767754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:39.767754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:39.767754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:39.767754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:39.767754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:39.767754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:39.767754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 13 21:38:39.767754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:39.767754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 258/518] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:39.767754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550a31c-5cca-4527-a670-9cb87476569f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:39.801515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c6e94-3b7e-44e3-b6a0-32f794ac1c44 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:39.834104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517594e-86c0-43a7-9f36-990f90949fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:39.834778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517594e-86c0-43a7-9f36-990f90949fb2 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:39.835065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517594e-86c0-43a7-9f36-990f90949fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:39.835372 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517594e-86c0-43a7-9f36-990f90949fb2 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:39.836084 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517594e-86c0-43a7-9f36-990f90949fb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:39.836392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517594e-86c0-43a7-9f36-990f90949fb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:39.837776 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517594e-86c0-43a7-9f36-990f90949fb2 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:39.848527 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 261/519] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:39.849300 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517594e-86c0-43a7-9f36-990f90949fb2 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:39.860274 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550a31c-5cca-4527-a670-9cb87476569f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:39.893570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ac694-2de2-4445-84ec-0d62de127d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:39.894320 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ac694-2de2-4445-84ec-0d62de127d1d None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:39.894618 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ac694-2de2-4445-84ec-0d62de127d1d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:39.894922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ac694-2de2-4445-84ec-0d62de127d1d None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:39.895631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ac694-2de2-4445-84ec-0d62de127d1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:39.895962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ac694-2de2-4445-84ec-0d62de127d1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:39.897324 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ac694-2de2-4445-84ec-0d62de127d1d None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:39.907982 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 259/520] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:39.908738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ac694-2de2-4445-84ec-0d62de127d1d None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:39.944699 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517594e-86c0-43a7-9f36-990f90949fb2 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:39.977791 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee022-cd4d-40f9-816b-35f7af7c61f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:39.978486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee022-cd4d-40f9-816b-35f7af7c61f7 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:39.978793 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee022-cd4d-40f9-816b-35f7af7c61f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:39.979069 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee022-cd4d-40f9-816b-35f7af7c61f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:39.989811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dee022-cd4d-40f9-816b-35f7af7c61f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:39.991320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dee022-cd4d-40f9-816b-35f7af7c61f7 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-96dee022-cd4d-40f9-816b-35f7af7c61f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:40.007691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:40.008718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:40.014309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:40.014309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:40.014309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:40.014309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:40.014309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 262/521] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:40.014309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee022-cd4d-40f9-816b-35f7af7c61f7 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:40.021880 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ac694-2de2-4445-84ec-0d62de127d1d None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:40.054292 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc00f4-732f-48ab-86c9-334f82d5783c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:40.055093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc00f4-732f-48ab-86c9-334f82d5783c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:40.055375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc00f4-732f-48ab-86c9-334f82d5783c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:40.055640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc00f4-732f-48ab-86c9-334f82d5783c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:40.065634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc00f4-732f-48ab-86c9-334f82d5783c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:40.067107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc00f4-732f-48ab-86c9-334f82d5783c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-88fc00f4-732f-48ab-86c9-334f82d5783c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:40.083686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:40.084579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:40.085460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:40.085460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:40.085460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:40.085460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:40.085460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 260/522] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:40.085718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc00f4-732f-48ab-86c9-334f82d5783c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:40.110344 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee022-cd4d-40f9-816b-35f7af7c61f7 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:40.141433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324181d-1bef-4245-9801-6796962f86b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:40.142089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324181d-1bef-4245-9801-6796962f86b9 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:40.142374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324181d-1bef-4245-9801-6796962f86b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:40.142651 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324181d-1bef-4245-9801-6796962f86b9 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:40.143506 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324181d-1bef-4245-9801-6796962f86b9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:40.143815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324181d-1bef-4245-9801-6796962f86b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:40.145247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324181d-1bef-4245-9801-6796962f86b9 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:40.162762 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 263/523] 10.222.0.35 () {56 vars in 1139 bytes} [Fri May 13 21:38:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:40.163551 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324181d-1bef-4245-9801-6796962f86b9 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:40.181802 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc00f4-732f-48ab-86c9-334f82d5783c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:40.213917 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddbf4a-0615-4f15-9d8a-abbf4ff675d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:40.214722 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddbf4a-0615-4f15-9d8a-abbf4ff675d0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:40.215030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddbf4a-0615-4f15-9d8a-abbf4ff675d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:40.215310 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddbf4a-0615-4f15-9d8a-abbf4ff675d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:40.222274 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddbf4a-0615-4f15-9d8a-abbf4ff675d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:40.223764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddbf4a-0615-4f15-9d8a-abbf4ff675d0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-1dddbf4a-0615-4f15-9d8a-abbf4ff675d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:40.238141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:40.239061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:40.239907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:40.239907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:40.239907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:40.239907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:40.239907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:40.239907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:40.239907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:40.239907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:40.239907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 261/524] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:40.240520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddbf4a-0615-4f15-9d8a-abbf4ff675d0 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:40.260090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324181d-1bef-4245-9801-6796962f86b9 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:40.294170 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131aacf-8f23-4434-b540-fbdb13951ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:40.294863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131aacf-8f23-4434-b540-fbdb13951ac9 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:40.295150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131aacf-8f23-4434-b540-fbdb13951ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:40.295480 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131aacf-8f23-4434-b540-fbdb13951ac9 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:40.296195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6131aacf-8f23-4434-b540-fbdb13951ac9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:40.297645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6131aacf-8f23-4434-b540-fbdb13951ac9 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:40.316654 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 264/525] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:38:40 2022] POST /identity/v3/users => generated 247 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:38:40.317469 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131aacf-8f23-4434-b540-fbdb13951ac9 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:40.346254 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddbf4a-0615-4f15-9d8a-abbf4ff675d0 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:40.387892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe499e1c-8a2a-45a5-aef3-bf13095ae08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:40.388737 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe499e1c-8a2a-45a5-aef3-bf13095ae08f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:40.389021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe499e1c-8a2a-45a5-aef3-bf13095ae08f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:40.389857 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe499e1c-8a2a-45a5-aef3-bf13095ae08f None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:40.390653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe499e1c-8a2a-45a5-aef3-bf13095ae08f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:40.390991 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe499e1c-8a2a-45a5-aef3-bf13095ae08f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:40.392432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe499e1c-8a2a-45a5-aef3-bf13095ae08f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:40.407089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 262/526] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:40.409040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe499e1c-8a2a-45a5-aef3-bf13095ae08f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:40.426129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131aacf-8f23-4434-b540-fbdb13951ac9 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:40.462334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3f7b7-446f-4e09-8c3a-b4e73203c153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:40.463065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3f7b7-446f-4e09-8c3a-b4e73203c153 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:40.463355 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3f7b7-446f-4e09-8c3a-b4e73203c153 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:40.463632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3f7b7-446f-4e09-8c3a-b4e73203c153 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:40.464368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3f7b7-446f-4e09-8c3a-b4e73203c153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:40.464684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3f7b7-446f-4e09-8c3a-b4e73203c153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:40.466106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3f7b7-446f-4e09-8c3a-b4e73203c153 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:40.483285 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 265/527] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:40.484544 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3f7b7-446f-4e09-8c3a-b4e73203c153 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:40.513890 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe499e1c-8a2a-45a5-aef3-bf13095ae08f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:40.549905 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33dece-1529-4247-9915-69793217a8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:40.550553 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33dece-1529-4247-9915-69793217a8e0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:40.550881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33dece-1529-4247-9915-69793217a8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:40.551158 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33dece-1529-4247-9915-69793217a8e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:40.594605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3f7b7-446f-4e09-8c3a-b4e73203c153 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:40.618030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c33dece-1529-4247-9915-69793217a8e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:40.619529 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c33dece-1529-4247-9915-69793217a8e0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:40.628824 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619d158-c88b-4e57-bc18-76b51650ad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:40.629482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619d158-c88b-4e57-bc18-76b51650ad51 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:40.629766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619d158-c88b-4e57-bc18-76b51650ad51 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:40.630043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619d158-c88b-4e57-bc18-76b51650ad51 None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:40.630857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619d158-c88b-4e57-bc18-76b51650ad51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:40.631172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619d158-c88b-4e57-bc18-76b51650ad51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:40.632614 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619d158-c88b-4e57-bc18-76b51650ad51 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:40.651279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 266/528] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:40.652119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619d158-c88b-4e57-bc18-76b51650ad51 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:40.721877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 263/529] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:40.726642 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33dece-1529-4247-9915-69793217a8e0 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:40.749067 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619d158-c88b-4e57-bc18-76b51650ad51 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:40.782407 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c3e98-b508-411f-81b0-e9b234e09d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:40.783143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c3e98-b508-411f-81b0-e9b234e09d2b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:40.783423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c3e98-b508-411f-81b0-e9b234e09d2b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:40.783694 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c3e98-b508-411f-81b0-e9b234e09d2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:40.790043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c3e98-b508-411f-81b0-e9b234e09d2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:40.791510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c3e98-b508-411f-81b0-e9b234e09d2b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-569c3e98-b508-411f-81b0-e9b234e09d2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:40.805278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:40.812177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:40.813293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:40.813293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:40.813293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:40.813293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:40.813293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:40.813293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:40.813293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:40.813293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:40.813293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 267/530] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:40.813293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c3e98-b508-411f-81b0-e9b234e09d2b None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:40.823294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33dece-1529-4247-9915-69793217a8e0 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:40.856526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01b5d9-4a6c-4dca-8e29-c12d3c68b2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:40.857313 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01b5d9-4a6c-4dca-8e29-c12d3c68b2ad None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:40.857596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01b5d9-4a6c-4dca-8e29-c12d3c68b2ad None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:40.857909 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01b5d9-4a6c-4dca-8e29-c12d3c68b2ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:40.865150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01b5d9-4a6c-4dca-8e29-c12d3c68b2ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:40.866615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01b5d9-4a6c-4dca-8e29-c12d3c68b2ad None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-af01b5d9-4a6c-4dca-8e29-c12d3c68b2ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:40.881906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:40.882922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:40.883768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:40.883768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:40.883768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:40.883768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:40.883768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:40.883768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:40.883768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:40.883768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:40.887021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 264/531] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:40.888729 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01b5d9-4a6c-4dca-8e29-c12d3c68b2ad None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:40.910235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c3e98-b508-411f-81b0-e9b234e09d2b None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:40.942915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f29fbf-d10d-45c8-b7e4-af9a694f8f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:40.943866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f29fbf-d10d-45c8-b7e4-af9a694f8f55 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:40.944348 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f29fbf-d10d-45c8-b7e4-af9a694f8f55 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:40.944821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f29fbf-d10d-45c8-b7e4-af9a694f8f55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:40.945877 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f29fbf-d10d-45c8-b7e4-af9a694f8f55 None admin] RBAC: Adding query filter params (role.id=deb38f497be14e78ae21e8dceca3772f, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=86c4a81ce97b4fd78889fae6c36bf6cb) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:40.946387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f29fbf-d10d-45c8-b7e4-af9a694f8f55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:40.948060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f29fbf-d10d-45c8-b7e4-af9a694f8f55 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:40.954111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 268/532] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:40 2022] GET /identity/v3/role_assignments?user.id=86c4a81ce97b4fd78889fae6c36bf6cb&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=deb38f497be14e78ae21e8dceca3772f => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:40.954985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f29fbf-d10d-45c8-b7e4-af9a694f8f55 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:41.002205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01b5d9-4a6c-4dca-8e29-c12d3c68b2ad None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:41.042898 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1f420-8648-4919-b3c8-b0a0cbd81d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:41.043734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1f420-8648-4919-b3c8-b0a0cbd81d13 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:41.044094 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1f420-8648-4919-b3c8-b0a0cbd81d13 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:41.044440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1f420-8648-4919-b3c8-b0a0cbd81d13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:41.059890 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1f420-8648-4919-b3c8-b0a0cbd81d13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:41.061766 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1f420-8648-4919-b3c8-b0a0cbd81d13 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-c3a1f420-8648-4919-b3c8-b0a0cbd81d13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:41.081025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:41.082020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:41.082902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:41.082902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:41.082902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:41.082902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:41.089620 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 265/533] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:41.090919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1f420-8648-4919-b3c8-b0a0cbd81d13 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:41.097251 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f29fbf-d10d-45c8-b7e4-af9a694f8f55 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:41.143397 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafba402-5c75-43d1-88d3-fe5840cc4566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:41.144208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafba402-5c75-43d1-88d3-fe5840cc4566 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:41.144509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafba402-5c75-43d1-88d3-fe5840cc4566 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:41.144787 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafba402-5c75-43d1-88d3-fe5840cc4566 None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:41.145888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafba402-5c75-43d1-88d3-fe5840cc4566 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:41.146209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafba402-5c75-43d1-88d3-fe5840cc4566 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:41.147716 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafba402-5c75-43d1-88d3-fe5840cc4566 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:41.172151 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 269/534] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:41.173059 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafba402-5c75-43d1-88d3-fe5840cc4566 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:41.214213 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1f420-8648-4919-b3c8-b0a0cbd81d13 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:41.248280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7e474-0df0-4937-b1ef-929698f12abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:41.249115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7e474-0df0-4937-b1ef-929698f12abe None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:41.249502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7e474-0df0-4937-b1ef-929698f12abe None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:41.249877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7e474-0df0-4937-b1ef-929698f12abe None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:41.250794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7e474-0df0-4937-b1ef-929698f12abe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:41.251206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7e474-0df0-4937-b1ef-929698f12abe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:41.253006 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7e474-0df0-4937-b1ef-929698f12abe None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:41.276368 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 266/535] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:41.277365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7e474-0df0-4937-b1ef-929698f12abe None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:41.280902 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafba402-5c75-43d1-88d3-fe5840cc4566 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:41.316052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b9c56-3116-43c9-8833-09469c869547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:41.316705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b9c56-3116-43c9-8833-09469c869547 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:41.316993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b9c56-3116-43c9-8833-09469c869547 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:41.317268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b9c56-3116-43c9-8833-09469c869547 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:41.317985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b9c56-3116-43c9-8833-09469c869547 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:41.318288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b9c56-3116-43c9-8833-09469c869547 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:41.319709 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b9c56-3116-43c9-8833-09469c869547 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:41.331626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 270/536] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:41.332564 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b9c56-3116-43c9-8833-09469c869547 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:41.379012 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7e474-0df0-4937-b1ef-929698f12abe None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:41.413007 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b874f-7941-4d75-9af4-e9cb0d24aa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:41.413661 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b874f-7941-4d75-9af4-e9cb0d24aa42 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:41.413948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b874f-7941-4d75-9af4-e9cb0d24aa42 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:41.414231 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b874f-7941-4d75-9af4-e9cb0d24aa42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:41.415090 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b874f-7941-4d75-9af4-e9cb0d24aa42 None admin] RBAC: Adding query filter params (role.id=deb38f497be14e78ae21e8dceca3772f, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=cad57b046f3a43978d1c98bcc1a76d9d) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:41.415405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b874f-7941-4d75-9af4-e9cb0d24aa42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:41.416846 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b874f-7941-4d75-9af4-e9cb0d24aa42 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:41.423417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 267/537] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:41 2022] GET /identity/v3/role_assignments?user.id=cad57b046f3a43978d1c98bcc1a76d9d&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=deb38f497be14e78ae21e8dceca3772f => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:41.424228 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b874f-7941-4d75-9af4-e9cb0d24aa42 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:41.442137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b9c56-3116-43c9-8833-09469c869547 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:41.516280 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf4638-06b8-483f-b940-c408b501ab5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:41.516943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf4638-06b8-483f-b940-c408b501ab5a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:41.517227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf4638-06b8-483f-b940-c408b501ab5a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:41.517504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf4638-06b8-483f-b940-c408b501ab5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:41.518341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf4638-06b8-483f-b940-c408b501ab5a None admin] RBAC: Adding query filter params (role.id=deb38f497be14e78ae21e8dceca3772f, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=316ef43c6c304bd3a27e085bf4da8a5e) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:41.518656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf4638-06b8-483f-b940-c408b501ab5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:41.520181 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf4638-06b8-483f-b940-c408b501ab5a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:41.531617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b874f-7941-4d75-9af4-e9cb0d24aa42 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:41.534069 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 271/538] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:41 2022] GET /identity/v3/role_assignments?user.id=316ef43c6c304bd3a27e085bf4da8a5e&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=deb38f497be14e78ae21e8dceca3772f => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:41.590628 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe857239-03aa-45ca-aeb2-3b381ca12199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:41.591403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe857239-03aa-45ca-aeb2-3b381ca12199 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:41.591688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe857239-03aa-45ca-aeb2-3b381ca12199 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:41.591971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe857239-03aa-45ca-aeb2-3b381ca12199 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:41.606263 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe857239-03aa-45ca-aeb2-3b381ca12199 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:41.607851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe857239-03aa-45ca-aeb2-3b381ca12199 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-fe857239-03aa-45ca-aeb2-3b381ca12199 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:41.624373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:41.626401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:41.628719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:41.628719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:41.628719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:41.628719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:41.628719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:41.628719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:41.628719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:41.628719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:41.628719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 268/539] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:41.631817 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf4638-06b8-483f-b940-c408b501ab5a None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:41.787860 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf4638-06b8-483f-b940-c408b501ab5a None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:41.830975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0ba2c-3aa8-4f15-a6f4-8b228209b1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:41.831630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0ba2c-3aa8-4f15-a6f4-8b228209b1a4 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:41.831898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0ba2c-3aa8-4f15-a6f4-8b228209b1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:41.832170 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0ba2c-3aa8-4f15-a6f4-8b228209b1a4 None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:41.832952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0ba2c-3aa8-4f15-a6f4-8b228209b1a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:41.833257 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0ba2c-3aa8-4f15-a6f4-8b228209b1a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:41.834688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0ba2c-3aa8-4f15-a6f4-8b228209b1a4 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:41.848452 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 272/540] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:41.853888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe857239-03aa-45ca-aeb2-3b381ca12199 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:41.958301 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe857239-03aa-45ca-aeb2-3b381ca12199 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:41.997783 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae74ec-b613-4908-91dd-9165588f61ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:41.999060 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae74ec-b613-4908-91dd-9165588f61ec None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:41.999352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae74ec-b613-4908-91dd-9165588f61ec None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:41.999627 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae74ec-b613-4908-91dd-9165588f61ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:42.000453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae74ec-b613-4908-91dd-9165588f61ec None admin] RBAC: Adding query filter params (role.id=deb38f497be14e78ae21e8dceca3772f, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=c4df058606c448a1bb757020e869ac33) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:42.000762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae74ec-b613-4908-91dd-9165588f61ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:42.002206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae74ec-b613-4908-91dd-9165588f61ec None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:42.008573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 269/541] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:41 2022] GET /identity/v3/role_assignments?user.id=c4df058606c448a1bb757020e869ac33&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=deb38f497be14e78ae21e8dceca3772f => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:42.625144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3ffc7-91f8-42ad-beba-5e30dd9c982f None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:42.625529 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3ffc7-91f8-42ad-beba-5e30dd9c982f None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:42.625751 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3ffc7-91f8-42ad-beba-5e30dd9c982f None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:42.627362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 273/542] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:42.633090 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17d5ed-3987-4ad3-991d-e000807171dd None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:42.633608 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17d5ed-3987-4ad3-991d-e000807171dd None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:42.634091 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17d5ed-3987-4ad3-991d-e000807171dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:42.656213 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-8c17d5ed-3987-4ad3-991d-e000807171dd None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:42.657757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8c17d5ed-3987-4ad3-991d-e000807171dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:42.757329 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 270/543] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:42.792113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3ffc7-91f8-42ad-beba-5e30dd9c982f None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:42.926844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3ffc7-91f8-42ad-beba-5e30dd9c982f None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:42.971376 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d9777-6ab9-45a6-97d7-df2a2fb1ce24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:42.972165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d9777-6ab9-45a6-97d7-df2a2fb1ce24 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:42.972447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d9777-6ab9-45a6-97d7-df2a2fb1ce24 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:42.972721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d9777-6ab9-45a6-97d7-df2a2fb1ce24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:42.986873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d9777-6ab9-45a6-97d7-df2a2fb1ce24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-1f5d9777-6ab9-45a6-97d7-df2a2fb1ce24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:42.994532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:42.995441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:42.996250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:42.996250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:42.996250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:42.996250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:42.996250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:42.996250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:42.996250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:42.996250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:42.996250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:42.996250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:42.996250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:42.996250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:43.000221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 274/544] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:43.001467 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17d5ed-3987-4ad3-991d-e000807171dd None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:43.144466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17d5ed-3987-4ad3-991d-e000807171dd None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:43.208438 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08258e3-9bf6-4394-9a27-25149fd62ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:43.209134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08258e3-9bf6-4394-9a27-25149fd62ae9 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:43.209422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08258e3-9bf6-4394-9a27-25149fd62ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:43.209692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08258e3-9bf6-4394-9a27-25149fd62ae9 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:43.210463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08258e3-9bf6-4394-9a27-25149fd62ae9 None admin] RBAC: Adding query filter params (name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:43.210806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08258e3-9bf6-4394-9a27-25149fd62ae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:43.212256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08258e3-9bf6-4394-9a27-25149fd62ae9 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:43.226603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 271/545] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:43.232047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d9777-6ab9-45a6-97d7-df2a2fb1ce24 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:43.353742 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d9777-6ab9-45a6-97d7-df2a2fb1ce24 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:43.390205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ad49c-d0eb-4fe5-9f7f-29782838ada7 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:43.390512 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ad49c-d0eb-4fe5-9f7f-29782838ada7 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:43.390800 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ad49c-d0eb-4fe5-9f7f-29782838ada7 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:43.392154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 272/546] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:43.397160 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7f781-90d1-46d0-a99b-8e07fb034383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:43.397399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e153ae3-77b3-4e5c-a875-d27db1c26a22 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:43.397634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e153ae3-77b3-4e5c-a875-d27db1c26a22 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:43.397922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e153ae3-77b3-4e5c-a875-d27db1c26a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:43.398026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7f781-90d1-46d0-a99b-8e07fb034383 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:43.398227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7f781-90d1-46d0-a99b-8e07fb034383 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:43.398496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7f781-90d1-46d0-a99b-8e07fb034383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:43.416526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7f781-90d1-46d0-a99b-8e07fb034383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:43.418013 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7f781-90d1-46d0-a99b-8e07fb034383 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:43.431442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-3e153ae3-77b3-4e5c-a875-d27db1c26a22 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:43.432644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-3e153ae3-77b3-4e5c-a875-d27db1c26a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-2ef7f781-90d1-46d0-a99b-8e07fb034383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:43.444298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:43.445121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:43.445960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:43.445960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:43.445960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:43.445960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:43.445960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 275/547] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:43.451003 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7f781-90d1-46d0-a99b-8e07fb034383 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:43.549963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 273/548] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:43.575579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e153ae3-77b3-4e5c-a875-d27db1c26a22 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:43.577987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7f781-90d1-46d0-a99b-8e07fb034383 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:43.615903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0c0ea-103b-4e1b-ad9f-a93036529167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:43.616583 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0c0ea-103b-4e1b-ad9f-a93036529167 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:43.616866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0c0ea-103b-4e1b-ad9f-a93036529167 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:43.617138 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0c0ea-103b-4e1b-ad9f-a93036529167 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:43.617865 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0c0ea-103b-4e1b-ad9f-a93036529167 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:43.618169 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0c0ea-103b-4e1b-ad9f-a93036529167 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:43.619603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0c0ea-103b-4e1b-ad9f-a93036529167 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:43.649416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 276/549] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:43.665069 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0c0ea-103b-4e1b-ad9f-a93036529167 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:43.707082 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e153ae3-77b3-4e5c-a875-d27db1c26a22 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:43.740406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4dcb0-ff7b-47f8-84b7-b2d4c3795f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:43.741205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4dcb0-ff7b-47f8-84b7-b2d4c3795f36 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:43.741490 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4dcb0-ff7b-47f8-84b7-b2d4c3795f36 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:43.741763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4dcb0-ff7b-47f8-84b7-b2d4c3795f36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:43.754067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4dcb0-ff7b-47f8-84b7-b2d4c3795f36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-67a4dcb0-ff7b-47f8-84b7-b2d4c3795f36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:43.762445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:43.763405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:43.764770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:43.764770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:43.764770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:43.764770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:43.764770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:43.764770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:43.764770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:43.764770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:43.764770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:43.764770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:43.764770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:43.764770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:43.764770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 274/550] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:43.770625 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4dcb0-ff7b-47f8-84b7-b2d4c3795f36 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:43.863364 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0c0ea-103b-4e1b-ad9f-a93036529167 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:43.925899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ec788-35fd-4719-8728-c5deb84166d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:43.927071 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ec788-35fd-4719-8728-c5deb84166d9 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:43.928738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ec788-35fd-4719-8728-c5deb84166d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:43.929456 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ec788-35fd-4719-8728-c5deb84166d9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:43.944905 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4dcb0-ff7b-47f8-84b7-b2d4c3795f36 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:43.947704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ec788-35fd-4719-8728-c5deb84166d9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:43.949376 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ec788-35fd-4719-8728-c5deb84166d9 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-4f4ec788-35fd-4719-8728-c5deb84166d9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:43.979044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:43.979980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:43.980879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:43.980879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:43.980879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:43.980879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:43.980879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:43.980879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:43.980879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:43.980879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:43.980879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 21:38:43.980879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:43.991857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 277/551] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:38:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:43.996057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ec788-35fd-4719-8728-c5deb84166d9 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:44.016505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f9456-e422-452f-a5b5-67c9c2e68c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:44.017886 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f9456-e422-452f-a5b5-67c9c2e68c4d None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:44.018384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f9456-e422-452f-a5b5-67c9c2e68c4d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:44.019061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f9456-e422-452f-a5b5-67c9c2e68c4d None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:44.021466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f9456-e422-452f-a5b5-67c9c2e68c4d None admin] RBAC: Adding query filter params (name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:44.021989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f9456-e422-452f-a5b5-67c9c2e68c4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:44.024431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f9456-e422-452f-a5b5-67c9c2e68c4d None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:44.035176 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 275/552] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:44.040870 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f9456-e422-452f-a5b5-67c9c2e68c4d None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:44.143435 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f9456-e422-452f-a5b5-67c9c2e68c4d None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:44.178891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893cfe5-f822-44b2-8604-3d4a06de5ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:44.180045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893cfe5-f822-44b2-8604-3d4a06de5ae8 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:44.180551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893cfe5-f822-44b2-8604-3d4a06de5ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:44.181025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893cfe5-f822-44b2-8604-3d4a06de5ae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:44.190220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ec788-35fd-4719-8728-c5deb84166d9 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:44.194446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893cfe5-f822-44b2-8604-3d4a06de5ae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:44.196369 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893cfe5-f822-44b2-8604-3d4a06de5ae8 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-c893cfe5-f822-44b2-8604-3d4a06de5ae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:44.216200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:44.217195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:44.218542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:44.218542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:44.218542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:44.218542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:44.220947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 276/553] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:44.226596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893cfe5-f822-44b2-8604-3d4a06de5ae8 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:44.253587 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb01fc0-f03f-495a-a437-a6ada02cfb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:44.254349 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb01fc0-f03f-495a-a437-a6ada02cfb92 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:44.254633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb01fc0-f03f-495a-a437-a6ada02cfb92 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:44.254931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb01fc0-f03f-495a-a437-a6ada02cfb92 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:44.255819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb01fc0-f03f-495a-a437-a6ada02cfb92 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:44.256128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb01fc0-f03f-495a-a437-a6ada02cfb92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:44.257610 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb01fc0-f03f-495a-a437-a6ada02cfb92 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:44.302052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 278/554] 10.222.0.35 () {56 vars in 1135 bytes} [Fri May 13 21:38:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:44.308500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb01fc0-f03f-495a-a437-a6ada02cfb92 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:44.344087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893cfe5-f822-44b2-8604-3d4a06de5ae8 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:44.412053 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d17c9-fbea-4777-be39-292fd67dfa7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:44.413394 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d17c9-fbea-4777-be39-292fd67dfa7a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:44.414132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d17c9-fbea-4777-be39-292fd67dfa7a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:44.414607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d17c9-fbea-4777-be39-292fd67dfa7a None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:44.416119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d17c9-fbea-4777-be39-292fd67dfa7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:44.416876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d17c9-fbea-4777-be39-292fd67dfa7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:44.418989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d17c9-fbea-4777-be39-292fd67dfa7a None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:44.435181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 277/555] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:44.441026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ccf42-80e5-4129-b651-4513f20f0fe9 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:44.441026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ccf42-80e5-4129-b651-4513f20f0fe9 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:44.441026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ccf42-80e5-4129-b651-4513f20f0fe9 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:44.443238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 278/556] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:44.444285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ccf42-80e5-4129-b651-4513f20f0fe9 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:44.469309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb01fc0-f03f-495a-a437-a6ada02cfb92 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:44.510221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029ac00-1640-4cb6-acc7-49e7ceb38ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:44.511270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029ac00-1640-4cb6-acc7-49e7ceb38ea7 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:44.511785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029ac00-1640-4cb6-acc7-49e7ceb38ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:44.512283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029ac00-1640-4cb6-acc7-49e7ceb38ea7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:44.568233 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ccf42-80e5-4129-b651-4513f20f0fe9 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:44.603700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e8ea7-e26c-459e-9301-8041634a365f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:44.604555 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e8ea7-e26c-459e-9301-8041634a365f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:44.605156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e8ea7-e26c-459e-9301-8041634a365f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:44.605475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e8ea7-e26c-459e-9301-8041634a365f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:44.611019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029ac00-1640-4cb6-acc7-49e7ceb38ea7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:44.613224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029ac00-1640-4cb6-acc7-49e7ceb38ea7 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:44.616874 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e8ea7-e26c-459e-9301-8041634a365f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:44.617805 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e8ea7-e26c-459e-9301-8041634a365f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-2c7e8ea7-e26c-459e-9301-8041634a365f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:44.632934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:44.633877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:44.634879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:44.634879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:44.634879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:44.634879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:44.634879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:44.634879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:44.634879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:44.634879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:44.634879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 13 21:38:44.634879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:44.639058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 279/557] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:44.640878 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2ddb2-9c06-4967-922d-28515d790c62 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:44.640878 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2ddb2-9c06-4967-922d-28515d790c62 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:44.641087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2ddb2-9c06-4967-922d-28515d790c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:44.660056 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-8af2ddb2-9c06-4967-922d-28515d790c62 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:44.661278 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8af2ddb2-9c06-4967-922d-28515d790c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:44.727927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 279/558] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 21:38:44.728989 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029ac00-1640-4cb6-acc7-49e7ceb38ea7 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:44.736885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 280/559] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:44.736885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9dcd8-124d-4b10-8ee4-51fd37854012 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:44.736885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9dcd8-124d-4b10-8ee4-51fd37854012 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:44.736885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9dcd8-124d-4b10-8ee4-51fd37854012 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:44.738846 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 281/560] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:44.740076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9dcd8-124d-4b10-8ee4-51fd37854012 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:44.824914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029ac00-1640-4cb6-acc7-49e7ceb38ea7 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:44.836559 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9dcd8-124d-4b10-8ee4-51fd37854012 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:44.855552 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8feec-6074-4a1b-8c11-5a4391161e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:44.856199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8feec-6074-4a1b-8c11-5a4391161e3a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:44.856477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8feec-6074-4a1b-8c11-5a4391161e3a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:44.856745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8feec-6074-4a1b-8c11-5a4391161e3a None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:44.857557 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8feec-6074-4a1b-8c11-5a4391161e3a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:44.858380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8feec-6074-4a1b-8c11-5a4391161e3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:44.859842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8feec-6074-4a1b-8c11-5a4391161e3a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:44.868404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6570e8-f9f8-4c5d-8f2e-37f63e376ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:44.869175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6570e8-f9f8-4c5d-8f2e-37f63e376ff4 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:44.869453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6570e8-f9f8-4c5d-8f2e-37f63e376ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:44.869723 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6570e8-f9f8-4c5d-8f2e-37f63e376ff4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:44.876452 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 280/561] 10.222.0.35 () {56 vars in 1137 bytes} [Fri May 13 21:38:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:44.877702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e2d5d-e5f3-4437-b0e8-de9f069512af None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:44.877990 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e2d5d-e5f3-4437-b0e8-de9f069512af None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:44.878262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e2d5d-e5f3-4437-b0e8-de9f069512af None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:44.879522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 281/562] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:44.880988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf02973-67d6-40ad-bad3-0bd9361c4d6a None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:44.881196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf02973-67d6-40ad-bad3-0bd9361c4d6a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:44.881473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf02973-67d6-40ad-bad3-0bd9361c4d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:44.885126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6570e8-f9f8-4c5d-8f2e-37f63e376ff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:44.886590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6570e8-f9f8-4c5d-8f2e-37f63e376ff4 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:44.896732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-dbf02973-67d6-40ad-bad3-0bd9361c4d6a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:44.897913 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-dbf02973-67d6-40ad-bad3-0bd9361c4d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-6f6570e8-f9f8-4c5d-8f2e-37f63e376ff4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:44.903561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:44.904385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:44.905814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:44.905814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:44.905814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:44.905814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:44.905814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 282/563] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:44.905814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6570e8-f9f8-4c5d-8f2e-37f63e376ff4 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:44.965485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 282/564] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:44.966336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf02973-67d6-40ad-bad3-0bd9361c4d6a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:45.004849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6570e8-f9f8-4c5d-8f2e-37f63e376ff4 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:45.036231 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8174e3-44a7-42df-b6ca-1264ce4b1a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:45.037025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8174e3-44a7-42df-b6ca-1264ce4b1a54 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:45.037309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8174e3-44a7-42df-b6ca-1264ce4b1a54 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:45.037589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8174e3-44a7-42df-b6ca-1264ce4b1a54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:45.047800 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8174e3-44a7-42df-b6ca-1264ce4b1a54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-8f8174e3-44a7-42df-b6ca-1264ce4b1a54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:45.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:45.056955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:45.057915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:45.057915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:45.057915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:45.057915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:45.057915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:45.057915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:45.057915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:45.057915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:45.057915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:45.057915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:45.057915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:45.057915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:45.057915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 283/565] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:45.058565 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f210f-a7c2-4888-b7a5-cb8b1bc02171 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:45.058565 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f210f-a7c2-4888-b7a5-cb8b1bc02171 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:45.058736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f210f-a7c2-4888-b7a5-cb8b1bc02171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:45.063839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf02973-67d6-40ad-bad3-0bd9361c4d6a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:45.074890 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-748f210f-a7c2-4888-b7a5-cb8b1bc02171 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:45.076082 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-748f210f-a7c2-4888-b7a5-cb8b1bc02171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:45.095745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7cc89-1c6e-44c2-9e72-525fbf974808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:45.096519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7cc89-1c6e-44c2-9e72-525fbf974808 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:45.096798 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7cc89-1c6e-44c2-9e72-525fbf974808 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:45.097068 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7cc89-1c6e-44c2-9e72-525fbf974808 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:45.106732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7cc89-1c6e-44c2-9e72-525fbf974808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:45.108707 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7cc89-1c6e-44c2-9e72-525fbf974808 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-aed7cc89-1c6e-44c2-9e72-525fbf974808 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:45.125492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:45.126340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:45.127395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:45.127395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:45.127395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:45.127395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:45.127395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 283/566] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:45.128309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7cc89-1c6e-44c2-9e72-525fbf974808 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:45.142844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 284/567] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:45.143808 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f210f-a7c2-4888-b7a5-cb8b1bc02171 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:45.224186 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7cc89-1c6e-44c2-9e72-525fbf974808 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:45.239452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f210f-a7c2-4888-b7a5-cb8b1bc02171 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:45.256252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779599b-9218-4036-a817-f4afec5ed595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:45.256845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779599b-9218-4036-a817-f4afec5ed595 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:45.257120 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779599b-9218-4036-a817-f4afec5ed595 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:45.257397 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779599b-9218-4036-a817-f4afec5ed595 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:45.258104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779599b-9218-4036-a817-f4afec5ed595 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:45.258405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779599b-9218-4036-a817-f4afec5ed595 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:45.259805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779599b-9218-4036-a817-f4afec5ed595 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:45.271107 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 284/568] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:45.271896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779599b-9218-4036-a817-f4afec5ed595 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:45.274269 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859440e-8807-49e2-8e33-642836a770b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:45.274975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859440e-8807-49e2-8e33-642836a770b3 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:45.275250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859440e-8807-49e2-8e33-642836a770b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:45.275704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859440e-8807-49e2-8e33-642836a770b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:45.285258 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859440e-8807-49e2-8e33-642836a770b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-a859440e-8807-49e2-8e33-642836a770b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:45.292770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:45.293738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:45.294626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:45.294626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:45.294626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:45.294626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:45.294626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:45.294626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:45.294626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:45.294626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:45.294626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:45.294626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:45.294626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:45.294626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:45.294626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 285/569] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:45.295324 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859440e-8807-49e2-8e33-642836a770b3 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:45.365071 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779599b-9218-4036-a817-f4afec5ed595 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:45.387845 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859440e-8807-49e2-8e33-642836a770b3 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:45.396727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5a3ae-c637-42c0-b0c7-7ab3fe4c07d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:45.397380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5a3ae-c637-42c0-b0c7-7ab3fe4c07d3 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:45.397656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5a3ae-c637-42c0-b0c7-7ab3fe4c07d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:45.397923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5a3ae-c637-42c0-b0c7-7ab3fe4c07d3 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:45.398653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5a3ae-c637-42c0-b0c7-7ab3fe4c07d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:45.398983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5a3ae-c637-42c0-b0c7-7ab3fe4c07d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:45.400344 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5a3ae-c637-42c0-b0c7-7ab3fe4c07d3 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:45.407807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 285/570] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:45.408561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5a3ae-c637-42c0-b0c7-7ab3fe4c07d3 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:45.420078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f22210-31ba-4bf3-a4e0-87f6198b8c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:45.420763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f22210-31ba-4bf3-a4e0-87f6198b8c3e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:45.421051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f22210-31ba-4bf3-a4e0-87f6198b8c3e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:45.421331 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f22210-31ba-4bf3-a4e0-87f6198b8c3e None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:45.422051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f22210-31ba-4bf3-a4e0-87f6198b8c3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:45.422357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f22210-31ba-4bf3-a4e0-87f6198b8c3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:45.423769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f22210-31ba-4bf3-a4e0-87f6198b8c3e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:45.435125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 286/571] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:45.436515 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f22210-31ba-4bf3-a4e0-87f6198b8c3e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:45.500349 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5a3ae-c637-42c0-b0c7-7ab3fe4c07d3 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:45.528793 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f22210-31ba-4bf3-a4e0-87f6198b8c3e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:45.531199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8463b-e037-4bf7-baa7-625640274274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:45.531978 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8463b-e037-4bf7-baa7-625640274274 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:45.532265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8463b-e037-4bf7-baa7-625640274274 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:45.532545 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8463b-e037-4bf7-baa7-625640274274 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:45.542314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8463b-e037-4bf7-baa7-625640274274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-68d8463b-e037-4bf7-baa7-625640274274 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:45.550580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:45.551567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:45.553315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:45.553315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:45.553315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:45.553315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:45.553315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:45.553315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:45.553315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:45.553315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:45.553315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:45.553315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:45.553315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:45.553315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:45.553315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 286/572] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:45.553315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8463b-e037-4bf7-baa7-625640274274 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:45.564434 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7f96a-6b25-4861-be2c-cdd7a01ec3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:45.565226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7f96a-6b25-4861-be2c-cdd7a01ec3c3 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:45.565504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7f96a-6b25-4861-be2c-cdd7a01ec3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:45.565774 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7f96a-6b25-4861-be2c-cdd7a01ec3c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:45.572304 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7f96a-6b25-4861-be2c-cdd7a01ec3c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:45.573742 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7f96a-6b25-4861-be2c-cdd7a01ec3c3 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-31e7f96a-6b25-4861-be2c-cdd7a01ec3c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:45.587054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:45.587903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:45.588803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:45.588803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:45.588803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:45.588803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:45.588803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:45.588803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:45.588803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:45.588803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:45.588803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 287/573] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:45.589367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7f96a-6b25-4861-be2c-cdd7a01ec3c3 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:45.648120 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8463b-e037-4bf7-baa7-625640274274 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:45.679274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfe741-ba92-41a5-828a-a0eb673eed6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:45.679928 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfe741-ba92-41a5-828a-a0eb673eed6e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:45.680261 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfe741-ba92-41a5-828a-a0eb673eed6e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:45.680538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfe741-ba92-41a5-828a-a0eb673eed6e None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:45.681267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dfe741-ba92-41a5-828a-a0eb673eed6e None admin] RBAC: Adding query filter params (name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:45.681572 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dfe741-ba92-41a5-828a-a0eb673eed6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:45.683024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dfe741-ba92-41a5-828a-a0eb673eed6e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:45.689200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7f96a-6b25-4861-be2c-cdd7a01ec3c3 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:45.692615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 287/574] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:45.693398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfe741-ba92-41a5-828a-a0eb673eed6e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:45.721908 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb375d-e452-4653-a04c-bf7d86d8b37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:45.722802 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb375d-e452-4653-a04c-bf7d86d8b37a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:45.723087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb375d-e452-4653-a04c-bf7d86d8b37a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:45.723365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb375d-e452-4653-a04c-bf7d86d8b37a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:45.733497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb375d-e452-4653-a04c-bf7d86d8b37a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:45.734970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb375d-e452-4653-a04c-bf7d86d8b37a None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-78eb375d-e452-4653-a04c-bf7d86d8b37a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:45.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:45.752290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:45.753450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:45.753450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:45.753450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:45.753450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:45.753450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 288/575] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:45.753747 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb375d-e452-4653-a04c-bf7d86d8b37a None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:45.790463 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfe741-ba92-41a5-828a-a0eb673eed6e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:45.821847 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff13e1-6c28-46f3-8ccb-dd36c078b612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:45.822625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff13e1-6c28-46f3-8ccb-dd36c078b612 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:45.822948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff13e1-6c28-46f3-8ccb-dd36c078b612 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:45.823225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff13e1-6c28-46f3-8ccb-dd36c078b612 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:45.830142 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff13e1-6c28-46f3-8ccb-dd36c078b612 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:45.831609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff13e1-6c28-46f3-8ccb-dd36c078b612 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-e5ff13e1-6c28-46f3-8ccb-dd36c078b612 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:45.845395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:45.846287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:45.847179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:45.847179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:45.847179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:45.847179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:45.847179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:45.847179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:45.847179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:45.847179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:45.847179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 288/576] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:45.847672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff13e1-6c28-46f3-8ccb-dd36c078b612 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:45.847748 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb375d-e452-4653-a04c-bf7d86d8b37a None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:45.879441 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3874c-3750-4e30-9cd5-d471ad22d5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:45.880095 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3874c-3750-4e30-9cd5-d471ad22d5d4 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:45.880380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3874c-3750-4e30-9cd5-d471ad22d5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:45.880651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3874c-3750-4e30-9cd5-d471ad22d5d4 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:45.881949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3874c-3750-4e30-9cd5-d471ad22d5d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:45.882255 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3874c-3750-4e30-9cd5-d471ad22d5d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:45.883644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3874c-3750-4e30-9cd5-d471ad22d5d4 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:45.891959 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 289/577] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:45.892539 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3874c-3750-4e30-9cd5-d471ad22d5d4 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:45.946793 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff13e1-6c28-46f3-8ccb-dd36c078b612 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:45.978898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b5ab0-4bed-4598-b3ed-28af83a4fcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:45.979551 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b5ab0-4bed-4598-b3ed-28af83a4fcb2 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:45.979829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b5ab0-4bed-4598-b3ed-28af83a4fcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:45.980104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b5ab0-4bed-4598-b3ed-28af83a4fcb2 None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:45.980934 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b5ab0-4bed-4598-b3ed-28af83a4fcb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:45.981246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b5ab0-4bed-4598-b3ed-28af83a4fcb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:45.982686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b5ab0-4bed-4598-b3ed-28af83a4fcb2 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:45.986333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3874c-3750-4e30-9cd5-d471ad22d5d4 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:45.994054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 289/578] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:45.994861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b5ab0-4bed-4598-b3ed-28af83a4fcb2 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:46.021935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f9cfe-e9d3-484e-9030-a458708cc86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:46.022725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f9cfe-e9d3-484e-9030-a458708cc86f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:46.023025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f9cfe-e9d3-484e-9030-a458708cc86f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:46.023299 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f9cfe-e9d3-484e-9030-a458708cc86f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:46.033810 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f9cfe-e9d3-484e-9030-a458708cc86f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:46.035267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f9cfe-e9d3-484e-9030-a458708cc86f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-d97f9cfe-e9d3-484e-9030-a458708cc86f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:46.052109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:46.053507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:46.054386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:46.054386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:46.054386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:46.054386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:46.054386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 290/579] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:46.054386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f9cfe-e9d3-484e-9030-a458708cc86f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:46.087878 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b5ab0-4bed-4598-b3ed-28af83a4fcb2 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:46.119354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdcc50-013a-467a-8dc5-089f50e3055e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:46.119999 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdcc50-013a-467a-8dc5-089f50e3055e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:46.120281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdcc50-013a-467a-8dc5-089f50e3055e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:46.120558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdcc50-013a-467a-8dc5-089f50e3055e None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:46.121268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdcc50-013a-467a-8dc5-089f50e3055e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:46.121571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdcc50-013a-467a-8dc5-089f50e3055e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:46.122948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdcc50-013a-467a-8dc5-089f50e3055e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:46.134205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 290/580] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:46.134991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdcc50-013a-467a-8dc5-089f50e3055e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:46.149938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f9cfe-e9d3-484e-9030-a458708cc86f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:46.183230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d27153-26da-428d-95b9-a0744bb9d5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:46.183880 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d27153-26da-428d-95b9-a0744bb9d5b4 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:46.184162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d27153-26da-428d-95b9-a0744bb9d5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:46.184432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d27153-26da-428d-95b9-a0744bb9d5b4 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:46.185200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d27153-26da-428d-95b9-a0744bb9d5b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:46.185504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d27153-26da-428d-95b9-a0744bb9d5b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:46.189817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d27153-26da-428d-95b9-a0744bb9d5b4 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:46.200505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 291/581] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:46.201302 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d27153-26da-428d-95b9-a0744bb9d5b4 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:46.227599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdcc50-013a-467a-8dc5-089f50e3055e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:46.258974 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855c3f9-b9d5-4e7b-acb3-907170a9f141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:46.259754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855c3f9-b9d5-4e7b-acb3-907170a9f141 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:46.260035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855c3f9-b9d5-4e7b-acb3-907170a9f141 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:46.260309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855c3f9-b9d5-4e7b-acb3-907170a9f141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:46.270308 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855c3f9-b9d5-4e7b-acb3-907170a9f141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:46.271792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855c3f9-b9d5-4e7b-acb3-907170a9f141 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-0855c3f9-b9d5-4e7b-acb3-907170a9f141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:46.288984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:46.289852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:46.290972 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:46.290972 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:46.290972 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:46.290972 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:46.290972 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 291/582] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:46.290972 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855c3f9-b9d5-4e7b-acb3-907170a9f141 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:46.296847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d27153-26da-428d-95b9-a0744bb9d5b4 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:46.328666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d3358-c53e-4afd-876f-cecf615248e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:46.329247 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d3358-c53e-4afd-876f-cecf615248e0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:46.329525 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d3358-c53e-4afd-876f-cecf615248e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:46.329797 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d3358-c53e-4afd-876f-cecf615248e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:46.330614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d3358-c53e-4afd-876f-cecf615248e0 None admin] RBAC: Adding query filter params (role.id=deb38f497be14e78ae21e8dceca3772f, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=4f9e07ed0a084c82b946aaf585de16fd) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:46.330947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d3358-c53e-4afd-876f-cecf615248e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:46.332364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d3358-c53e-4afd-876f-cecf615248e0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:46.337633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 292/583] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:46 2022] GET /identity/v3/role_assignments?user.id=4f9e07ed0a084c82b946aaf585de16fd&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=deb38f497be14e78ae21e8dceca3772f => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:46.338383 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d3358-c53e-4afd-876f-cecf615248e0 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:46.388483 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855c3f9-b9d5-4e7b-acb3-907170a9f141 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:46.425004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48c46e-3724-4eb1-9937-db7434069ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:46.425677 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48c46e-3724-4eb1-9937-db7434069ce7 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:46.425959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48c46e-3724-4eb1-9937-db7434069ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:46.426234 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48c46e-3724-4eb1-9937-db7434069ce7 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:46.426996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48c46e-3724-4eb1-9937-db7434069ce7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:46.427309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48c46e-3724-4eb1-9937-db7434069ce7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:46.428742 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48c46e-3724-4eb1-9937-db7434069ce7 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:46.445741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 292/584] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:46.445741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48c46e-3724-4eb1-9937-db7434069ce7 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:46.446022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d3358-c53e-4afd-876f-cecf615248e0 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:46.478002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9e534-2207-4097-9b3f-7d0db8606386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:46.478869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9e534-2207-4097-9b3f-7d0db8606386 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:46.479154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9e534-2207-4097-9b3f-7d0db8606386 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:46.479430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9e534-2207-4097-9b3f-7d0db8606386 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:46.487871 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9e534-2207-4097-9b3f-7d0db8606386 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:46.489350 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9e534-2207-4097-9b3f-7d0db8606386 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-d0f9e534-2207-4097-9b3f-7d0db8606386 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:46.504711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:46.505815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:46.506785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:46.506785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:46.506785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:46.506785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:46.506785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:46.506785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:46.506785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:46.506785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:46.506785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 13 21:38:46.506785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:46.510628 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 293/585] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:46.511555 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9e534-2207-4097-9b3f-7d0db8606386 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:46.547666 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48c46e-3724-4eb1-9937-db7434069ce7 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:46.583175 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd8c0e-bfe7-4ed7-819e-6e1933cdc0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:46.583980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd8c0e-bfe7-4ed7-819e-6e1933cdc0ad None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:46.584258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd8c0e-bfe7-4ed7-819e-6e1933cdc0ad None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:46.584536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd8c0e-bfe7-4ed7-819e-6e1933cdc0ad None admin] PATH_INFO: `/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/86c4a81ce97b4fd78889fae6c36bf6cb/roles/deb38f497be14e78ae21e8dceca3772f` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:46.598573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd8c0e-bfe7-4ed7-819e-6e1933cdc0ad None admin] RBAC: Authorizing `identity:create_grant(project_id=ac834e84b2c84261ade311a6dfbbeb20, user_id=86c4a81ce97b4fd78889fae6c36bf6cb, role_id=deb38f497be14e78ae21e8dceca3772f)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:46.600057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd8c0e-bfe7-4ed7-819e-6e1933cdc0ad None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:46.622738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 293/586] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:38:46 2022] PUT /identity/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/86c4a81ce97b4fd78889fae6c36bf6cb/roles/deb38f497be14e78ae21e8dceca3772f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:38:46.622738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd8c0e-bfe7-4ed7-819e-6e1933cdc0ad None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:46.623071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9e534-2207-4097-9b3f-7d0db8606386 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:46.653216 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2a682-664a-4ecf-bf31-302cdfae8870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:46.653986 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2a682-664a-4ecf-bf31-302cdfae8870 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:46.654346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2a682-664a-4ecf-bf31-302cdfae8870 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:46.654729 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2a682-664a-4ecf-bf31-302cdfae8870 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:46.655548 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2a682-664a-4ecf-bf31-302cdfae8870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:46.655920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2a682-664a-4ecf-bf31-302cdfae8870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:46.657557 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2a682-664a-4ecf-bf31-302cdfae8870 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:46.671322 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 294/587] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:46.672311 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2a682-664a-4ecf-bf31-302cdfae8870 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:46.721996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd8c0e-bfe7-4ed7-819e-6e1933cdc0ad None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:46.761594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece87a7-a54b-4242-89d1-8cc120a79b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:46.762675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece87a7-a54b-4242-89d1-8cc120a79b4e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:46.762996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece87a7-a54b-4242-89d1-8cc120a79b4e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:46.763275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece87a7-a54b-4242-89d1-8cc120a79b4e None admin] PATH_INFO: `/v3/users/nova` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:46.774114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece87a7-a54b-4242-89d1-8cc120a79b4e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:46.776165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece87a7-a54b-4242-89d1-8cc120a79b4e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:46.784209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2a682-664a-4ecf-bf31-302cdfae8870 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-9ece87a7-a54b-4242-89d1-8cc120a79b4e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:46.797104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:46.798087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:46.799045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:46.799045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:46.799045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:46.799045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:46.799045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:46.799045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:46.799045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:46.799045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:46.799045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 21:38:46.799045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:46.799045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 294/588] 10.222.0.35 () {56 vars in 1093 bytes} [Fri May 13 21:38:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:46.800127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece87a7-a54b-4242-89d1-8cc120a79b4e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:46.822414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8331ce8-ddfc-4fc1-b1ee-642cd2c47185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:46.823390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8331ce8-ddfc-4fc1-b1ee-642cd2c47185 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:46.823676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8331ce8-ddfc-4fc1-b1ee-642cd2c47185 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:46.823953 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8331ce8-ddfc-4fc1-b1ee-642cd2c47185 None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:46.824976 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8331ce8-ddfc-4fc1-b1ee-642cd2c47185 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:46.825601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8331ce8-ddfc-4fc1-b1ee-642cd2c47185 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:46.827731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8331ce8-ddfc-4fc1-b1ee-642cd2c47185 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:46.849425 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 295/589] 10.222.0.35 () {56 vars in 1137 bytes} [Fri May 13 21:38:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:46.850191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8331ce8-ddfc-4fc1-b1ee-642cd2c47185 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:46.921350 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece87a7-a54b-4242-89d1-8cc120a79b4e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:46.958472 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94679f-f86e-47ad-baa6-3048fa34d19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:46.959319 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94679f-f86e-47ad-baa6-3048fa34d19c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:46.959560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94679f-f86e-47ad-baa6-3048fa34d19c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:46.959841 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94679f-f86e-47ad-baa6-3048fa34d19c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:46.967273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da94679f-f86e-47ad-baa6-3048fa34d19c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:46.968422 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da94679f-f86e-47ad-baa6-3048fa34d19c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:46.980285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8331ce8-ddfc-4fc1-b1ee-642cd2c47185 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-da94679f-f86e-47ad-baa6-3048fa34d19c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:46.983825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:46.985327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:46.986473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:46.986473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:46.986473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:46.986473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:46.986473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:46.986473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:46.986473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:46.986473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:46.986473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 13 21:38:46.986473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:46.986473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 295/590] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:46.986473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94679f-f86e-47ad-baa6-3048fa34d19c None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:47.033328 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc34e8-aad6-43c7-89a0-8df57cfc42f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:47.034000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc34e8-aad6-43c7-89a0-8df57cfc42f5 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:47.034278 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc34e8-aad6-43c7-89a0-8df57cfc42f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:47.034552 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc34e8-aad6-43c7-89a0-8df57cfc42f5 None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:47.035548 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc34e8-aad6-43c7-89a0-8df57cfc42f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:47.036075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc34e8-aad6-43c7-89a0-8df57cfc42f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:47.038576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc34e8-aad6-43c7-89a0-8df57cfc42f5 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:47.057276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 296/591] 10.222.0.35 () {56 vars in 1133 bytes} [Fri May 13 21:38:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:47.058108 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc34e8-aad6-43c7-89a0-8df57cfc42f5 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:47.096854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94679f-f86e-47ad-baa6-3048fa34d19c None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:47.131880 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a87be1-5e99-4947-93e3-d5d649caeef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:47.133010 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a87be1-5e99-4947-93e3-d5d649caeef1 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:47.133504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a87be1-5e99-4947-93e3-d5d649caeef1 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:47.134008 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a87be1-5e99-4947-93e3-d5d649caeef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:47.146070 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a87be1-5e99-4947-93e3-d5d649caeef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:47.147848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a87be1-5e99-4947-93e3-d5d649caeef1 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:47.163008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc34e8-aad6-43c7-89a0-8df57cfc42f5 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-07a87be1-5e99-4947-93e3-d5d649caeef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:47.170675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:47.171630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:47.172556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:47.172556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:47.172556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:47.172556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:47.174879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 296/592] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:47.175972 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a87be1-5e99-4947-93e3-d5d649caeef1 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:47.210230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ccfba-9f3e-4fe5-ba1c-34abdadabb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:47.210732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ccfba-9f3e-4fe5-ba1c-34abdadabb2c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:47.211045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ccfba-9f3e-4fe5-ba1c-34abdadabb2c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:47.211366 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ccfba-9f3e-4fe5-ba1c-34abdadabb2c None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:47.212201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ccfba-9f3e-4fe5-ba1c-34abdadabb2c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:47.212504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ccfba-9f3e-4fe5-ba1c-34abdadabb2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:47.213919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ccfba-9f3e-4fe5-ba1c-34abdadabb2c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:47.231140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 297/593] 10.222.0.35 () {56 vars in 1139 bytes} [Fri May 13 21:38:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:47.231953 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ccfba-9f3e-4fe5-ba1c-34abdadabb2c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:47.281485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a87be1-5e99-4947-93e3-d5d649caeef1 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:47.316410 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4deb0e1-7e33-4f9d-b19a-0e6c0b2cb1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:47.317656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deb0e1-7e33-4f9d-b19a-0e6c0b2cb1d1 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:47.318151 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deb0e1-7e33-4f9d-b19a-0e6c0b2cb1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:47.318634 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deb0e1-7e33-4f9d-b19a-0e6c0b2cb1d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:47.328933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4deb0e1-7e33-4f9d-b19a-0e6c0b2cb1d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:47.330629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4deb0e1-7e33-4f9d-b19a-0e6c0b2cb1d1 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-d4deb0e1-7e33-4f9d-b19a-0e6c0b2cb1d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:47.348813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:47.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:47.350593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:47.350593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:47.350593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:47.350593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:47.352940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 297/594] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:47.353999 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4deb0e1-7e33-4f9d-b19a-0e6c0b2cb1d1 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:47.359601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ccfba-9f3e-4fe5-ba1c-34abdadabb2c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:47.395590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b24715-bb81-4520-a6d9-d2df24d34d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:47.396214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b24715-bb81-4520-a6d9-d2df24d34d55 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:47.396499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b24715-bb81-4520-a6d9-d2df24d34d55 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:47.396779 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b24715-bb81-4520-a6d9-d2df24d34d55 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:47.397506 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b24715-bb81-4520-a6d9-d2df24d34d55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:47.397813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b24715-bb81-4520-a6d9-d2df24d34d55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:47.399225 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b24715-bb81-4520-a6d9-d2df24d34d55 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:47.411507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 298/595] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:47.416778 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b24715-bb81-4520-a6d9-d2df24d34d55 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:47.458355 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4deb0e1-7e33-4f9d-b19a-0e6c0b2cb1d1 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:47.494064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af31df6-a115-4a53-8540-b5382b94b601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:47.495235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af31df6-a115-4a53-8540-b5382b94b601 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:47.495745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af31df6-a115-4a53-8540-b5382b94b601 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:47.496232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af31df6-a115-4a53-8540-b5382b94b601 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:47.507318 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af31df6-a115-4a53-8540-b5382b94b601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:47.509086 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af31df6-a115-4a53-8540-b5382b94b601 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:47.520806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b24715-bb81-4520-a6d9-d2df24d34d55 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-3af31df6-a115-4a53-8540-b5382b94b601 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:47.528057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:47.528971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:47.530389 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:47.530389 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:47.530389 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:47.530389 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:47.532712 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 298/596] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:47.533809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af31df6-a115-4a53-8540-b5382b94b601 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:47.557157 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6923da-19a1-4bf2-9523-ef1842f9c653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:47.557986 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6923da-19a1-4bf2-9523-ef1842f9c653 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:47.558282 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6923da-19a1-4bf2-9523-ef1842f9c653 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:47.558554 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6923da-19a1-4bf2-9523-ef1842f9c653 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:47.559404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6923da-19a1-4bf2-9523-ef1842f9c653 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:47.559755 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6923da-19a1-4bf2-9523-ef1842f9c653 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:47.561257 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6923da-19a1-4bf2-9523-ef1842f9c653 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:47.573647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 299/597] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:47.574574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6923da-19a1-4bf2-9523-ef1842f9c653 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:47.636631 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af31df6-a115-4a53-8540-b5382b94b601 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:47.676162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bb83d-f4a0-47ca-9c19-e62999e7b692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:47.677321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb83d-f4a0-47ca-9c19-e62999e7b692 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:47.677821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb83d-f4a0-47ca-9c19-e62999e7b692 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:47.678303 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb83d-f4a0-47ca-9c19-e62999e7b692 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:47.685521 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bb83d-f4a0-47ca-9c19-e62999e7b692 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:47.687287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bb83d-f4a0-47ca-9c19-e62999e7b692 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:47.693103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6923da-19a1-4bf2-9523-ef1842f9c653 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-7c7bb83d-f4a0-47ca-9c19-e62999e7b692 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:47.701985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:47.702899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:47.703780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:47.703780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:47.703780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:47.703780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:47.703780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:47.703780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:47.703780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:47.703780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:47.706215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 299/598] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:47.707305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bb83d-f4a0-47ca-9c19-e62999e7b692 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:47.737317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41869d-7d47-4db8-9d4b-8e9cc4fb4e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:47.738568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41869d-7d47-4db8-9d4b-8e9cc4fb4e01 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:47.739079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41869d-7d47-4db8-9d4b-8e9cc4fb4e01 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:47.739694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41869d-7d47-4db8-9d4b-8e9cc4fb4e01 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:47.741331 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41869d-7d47-4db8-9d4b-8e9cc4fb4e01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:47.741822 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41869d-7d47-4db8-9d4b-8e9cc4fb4e01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:47.743572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41869d-7d47-4db8-9d4b-8e9cc4fb4e01 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:47.758421 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 300/599] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:47.760071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41869d-7d47-4db8-9d4b-8e9cc4fb4e01 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:47.833472 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bb83d-f4a0-47ca-9c19-e62999e7b692 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:47.874494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41869d-7d47-4db8-9d4b-8e9cc4fb4e01 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:47.881079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb218f-fba8-401d-bd13-79da78179b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:47.881951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb218f-fba8-401d-bd13-79da78179b90 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:47.882306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb218f-fba8-401d-bd13-79da78179b90 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:47.882658 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb218f-fba8-401d-bd13-79da78179b90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:47.892296 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb218f-fba8-401d-bd13-79da78179b90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:47.894000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb218f-fba8-401d-bd13-79da78179b90 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-5fcb218f-fba8-401d-bd13-79da78179b90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:47.908977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:47.909889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:47.911196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:47.911196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:47.911196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:47.911196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:47.911196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:47.911196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:47.911196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:47.911196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:47.917273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 300/600] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:47.917273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb218f-fba8-401d-bd13-79da78179b90 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:47.920348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420687b2-73a0-432c-9389-c39250550654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:47.920847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420687b2-73a0-432c-9389-c39250550654 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:47.921131 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420687b2-73a0-432c-9389-c39250550654 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:47.921405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420687b2-73a0-432c-9389-c39250550654 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:47.922193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420687b2-73a0-432c-9389-c39250550654 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:47.922504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420687b2-73a0-432c-9389-c39250550654 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:47.923993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420687b2-73a0-432c-9389-c39250550654 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:47.942414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 301/601] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:47.943398 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420687b2-73a0-432c-9389-c39250550654 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:48.022695 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb218f-fba8-401d-bd13-79da78179b90 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:48.047958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420687b2-73a0-432c-9389-c39250550654 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:48.058098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96742e11-532a-4e95-9687-739463ff72cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:48.059186 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96742e11-532a-4e95-9687-739463ff72cd None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:48.059701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96742e11-532a-4e95-9687-739463ff72cd None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:48.060201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96742e11-532a-4e95-9687-739463ff72cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:48.067815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96742e11-532a-4e95-9687-739463ff72cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:48.069551 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96742e11-532a-4e95-9687-739463ff72cd None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:48.083160 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b88fcf-f616-4b01-ac2f-459f18471e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:48.083845 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b88fcf-f616-4b01-ac2f-459f18471e95 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:48.084104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b88fcf-f616-4b01-ac2f-459f18471e95 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:48.084384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b88fcf-f616-4b01-ac2f-459f18471e95 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:48.085171 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b88fcf-f616-4b01-ac2f-459f18471e95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:48.085481 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b88fcf-f616-4b01-ac2f-459f18471e95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-96742e11-532a-4e95-9687-739463ff72cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:48.086948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:48.087814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:48.088700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:48.088700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:48.088700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:48.088700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:48.088700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:48.088700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:48.088700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:48.088700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:48.089129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b88fcf-f616-4b01-ac2f-459f18471e95 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:48.091331 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 301/602] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:48.092477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96742e11-532a-4e95-9687-739463ff72cd None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:48.103493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 302/603] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:48.103493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b88fcf-f616-4b01-ac2f-459f18471e95 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:48.205300 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96742e11-532a-4e95-9687-739463ff72cd None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:48.210395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b88fcf-f616-4b01-ac2f-459f18471e95 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:48.239169 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec83ee-a471-41ce-b79c-50cccb190f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:48.240463 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec83ee-a471-41ce-b79c-50cccb190f40 None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:48.241025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec83ee-a471-41ce-b79c-50cccb190f40 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:48.241545 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec83ee-a471-41ce-b79c-50cccb190f40 None admin] PATH_INFO: `/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/cad57b046f3a43978d1c98bcc1a76d9d/roles/deb38f497be14e78ae21e8dceca3772f` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:48.243743 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d240f-0aa3-4734-93ea-48078351f0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:48.244501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d240f-0aa3-4734-93ea-48078351f0ff None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:48.244863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d240f-0aa3-4734-93ea-48078351f0ff None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:48.245186 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d240f-0aa3-4734-93ea-48078351f0ff None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:48.246618 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d240f-0aa3-4734-93ea-48078351f0ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:48.246618 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d240f-0aa3-4734-93ea-48078351f0ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:48.247891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d240f-0aa3-4734-93ea-48078351f0ff None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:48.259048 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec83ee-a471-41ce-b79c-50cccb190f40 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac834e84b2c84261ade311a6dfbbeb20, user_id=cad57b046f3a43978d1c98bcc1a76d9d, role_id=deb38f497be14e78ae21e8dceca3772f)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:48.260780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec83ee-a471-41ce-b79c-50cccb190f40 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:48.261799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 303/604] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:48.262674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d240f-0aa3-4734-93ea-48078351f0ff None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:48.281041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 302/605] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:38:48 2022] PUT /identity/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/cad57b046f3a43978d1c98bcc1a76d9d/roles/deb38f497be14e78ae21e8dceca3772f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:38:48.282345 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec83ee-a471-41ce-b79c-50cccb190f40 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:48.371609 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d240f-0aa3-4734-93ea-48078351f0ff None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:48.411893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d846865-8f70-42d7-9fdc-deaf96d829ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:48.412971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d846865-8f70-42d7-9fdc-deaf96d829ff None admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:48.413155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d846865-8f70-42d7-9fdc-deaf96d829ff None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:48.413557 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d846865-8f70-42d7-9fdc-deaf96d829ff None admin] PATH_INFO: `/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/316ef43c6c304bd3a27e085bf4da8a5e/roles/deb38f497be14e78ae21e8dceca3772f` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:48.456443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d846865-8f70-42d7-9fdc-deaf96d829ff None admin] RBAC: Authorizing `identity:create_grant(project_id=ac834e84b2c84261ade311a6dfbbeb20, user_id=316ef43c6c304bd3a27e085bf4da8a5e, role_id=deb38f497be14e78ae21e8dceca3772f)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:48.457840 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d846865-8f70-42d7-9fdc-deaf96d829ff None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:48.464804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec83ee-a471-41ce-b79c-50cccb190f40 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:48.505343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 304/606] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:38:48 2022] PUT /identity/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/316ef43c6c304bd3a27e085bf4da8a5e/roles/deb38f497be14e78ae21e8dceca3772f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:38:48.518252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203f711-e505-46bd-b3e1-e0107df56db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:48.519259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203f711-e505-46bd-b3e1-e0107df56db1 None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:48.519541 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203f711-e505-46bd-b3e1-e0107df56db1 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:48.519814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203f711-e505-46bd-b3e1-e0107df56db1 None admin] PATH_INFO: `/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/c4df058606c448a1bb757020e869ac33/roles/deb38f497be14e78ae21e8dceca3772f` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:48.546496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203f711-e505-46bd-b3e1-e0107df56db1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac834e84b2c84261ade311a6dfbbeb20, user_id=c4df058606c448a1bb757020e869ac33, role_id=deb38f497be14e78ae21e8dceca3772f)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:48.548231 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203f711-e505-46bd-b3e1-e0107df56db1 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:48.571812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 303/607] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:38:48 2022] PUT /identity/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/c4df058606c448a1bb757020e869ac33/roles/deb38f497be14e78ae21e8dceca3772f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:38:48.589610 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd625355-2d58-41ab-94d7-440c02553bde None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:48.589962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd625355-2d58-41ab-94d7-440c02553bde None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:48.590168 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd625355-2d58-41ab-94d7-440c02553bde None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:48.592255 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 305/608] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:48.598519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142e667-34e3-4c64-ae38-5f35ad048b4c None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:48.598920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142e667-34e3-4c64-ae38-5f35ad048b4c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:48.599200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142e667-34e3-4c64-ae38-5f35ad048b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:48.619104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e142e667-34e3-4c64-ae38-5f35ad048b4c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:48.620350 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e142e667-34e3-4c64-ae38-5f35ad048b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:48.702264 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 304/609] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:48.725429 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd625355-2d58-41ab-94d7-440c02553bde None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:48.838130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd625355-2d58-41ab-94d7-440c02553bde None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:48.876042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a569a-04cc-4f60-bf3d-97a76cb2e3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:48.876833 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a569a-04cc-4f60-bf3d-97a76cb2e3bb None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:48.877110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a569a-04cc-4f60-bf3d-97a76cb2e3bb None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:48.877629 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a569a-04cc-4f60-bf3d-97a76cb2e3bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:48.888558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a569a-04cc-4f60-bf3d-97a76cb2e3bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-946a569a-04cc-4f60-bf3d-97a76cb2e3bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:48.896586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:48.898919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:48.900956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:48.900956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:48.900956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:48.900956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:48.900956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:48.900956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:48.900956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:48.900956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:48.900956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:48.900956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:48.900956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:48.900956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:48.900956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 306/610] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:48.905232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142e667-34e3-4c64-ae38-5f35ad048b4c None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:49.021395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142e667-34e3-4c64-ae38-5f35ad048b4c None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:49.060221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a2496-ae53-4bb8-808b-bade51cf2829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:49.061048 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a2496-ae53-4bb8-808b-bade51cf2829 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:49.061208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a2496-ae53-4bb8-808b-bade51cf2829 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:49.061468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a2496-ae53-4bb8-808b-bade51cf2829 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:49.062210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a2496-ae53-4bb8-808b-bade51cf2829 None admin] RBAC: Adding query filter params (name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:49.062512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a2496-ae53-4bb8-808b-bade51cf2829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:49.063978 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a2496-ae53-4bb8-808b-bade51cf2829 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:49.078019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 305/611] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:49.083567 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a569a-04cc-4f60-bf3d-97a76cb2e3bb None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:49.149241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160538db-bd39-4b6b-be31-562c4d06259b None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:49.149571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160538db-bd39-4b6b-be31-562c4d06259b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:49.149779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160538db-bd39-4b6b-be31-562c4d06259b None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:49.151446 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 306/612] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:49.157297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486b5e3-6116-458a-b599-ffecaf7553d7 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:49.157586 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486b5e3-6116-458a-b599-ffecaf7553d7 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:49.157863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486b5e3-6116-458a-b599-ffecaf7553d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:49.179310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e486b5e3-6116-458a-b599-ffecaf7553d7 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:49.180497 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e486b5e3-6116-458a-b599-ffecaf7553d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:49.198746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a569a-04cc-4f60-bf3d-97a76cb2e3bb None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:49.244316 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067ec36-1a92-4393-811c-c89108a8c3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:49.245129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067ec36-1a92-4393-811c-c89108a8c3ec None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:49.245412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067ec36-1a92-4393-811c-c89108a8c3ec None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:49.245688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067ec36-1a92-4393-811c-c89108a8c3ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:49.249464 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 307/613] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:49.263293 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067ec36-1a92-4393-811c-c89108a8c3ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:49.270985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067ec36-1a92-4393-811c-c89108a8c3ec None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:49.273236 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486b5e3-6116-458a-b599-ffecaf7553d7 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-2067ec36-1a92-4393-811c-c89108a8c3ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:49.292992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:49.295690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:49.297869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:49.297869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:49.297869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:49.297869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:49.297869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 307/614] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:49.301491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067ec36-1a92-4393-811c-c89108a8c3ec None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:49.379614 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486b5e3-6116-458a-b599-ffecaf7553d7 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:49.414998 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe701e2-5e2e-48b1-9b33-9a928464fb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:49.415792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe701e2-5e2e-48b1-9b33-9a928464fb5e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:49.416071 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe701e2-5e2e-48b1-9b33-9a928464fb5e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:49.416344 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe701e2-5e2e-48b1-9b33-9a928464fb5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:49.417174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067ec36-1a92-4393-811c-c89108a8c3ec None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:49.427503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe701e2-5e2e-48b1-9b33-9a928464fb5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-ffe701e2-5e2e-48b1-9b33-9a928464fb5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:49.435096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:49.437971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:49.440035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:49.440035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:49.440035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:49.440035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:49.440035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:49.440035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:49.440035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:49.440035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:49.440035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:49.440035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:49.440035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:49.440035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:49.440035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 308/615] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:49.447880 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe701e2-5e2e-48b1-9b33-9a928464fb5e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:49.460520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16df59-7dcc-40ef-89af-1fa9a3ff3ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:49.461203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16df59-7dcc-40ef-89af-1fa9a3ff3ced None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:49.461486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16df59-7dcc-40ef-89af-1fa9a3ff3ced None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:49.461759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16df59-7dcc-40ef-89af-1fa9a3ff3ced None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:49.462494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16df59-7dcc-40ef-89af-1fa9a3ff3ced None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:49.462842 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16df59-7dcc-40ef-89af-1fa9a3ff3ced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:49.464236 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16df59-7dcc-40ef-89af-1fa9a3ff3ced None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:49.479612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 308/616] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:49.491757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16df59-7dcc-40ef-89af-1fa9a3ff3ced None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:49.619325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe701e2-5e2e-48b1-9b33-9a928464fb5e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:49.656624 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346e5b2-7428-4e0a-a82e-1d6bde00df55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:49.657620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346e5b2-7428-4e0a-a82e-1d6bde00df55 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:49.658138 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346e5b2-7428-4e0a-a82e-1d6bde00df55 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:49.658821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346e5b2-7428-4e0a-a82e-1d6bde00df55 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:49.660489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346e5b2-7428-4e0a-a82e-1d6bde00df55 None admin] RBAC: Adding query filter params (name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:49.661691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346e5b2-7428-4e0a-a82e-1d6bde00df55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:49.663359 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346e5b2-7428-4e0a-a82e-1d6bde00df55 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:49.680057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 309/617] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:49.686612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346e5b2-7428-4e0a-a82e-1d6bde00df55 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:49.706148 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16df59-7dcc-40ef-89af-1fa9a3ff3ced None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:49.788347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b5483-e9cb-481d-8227-dba6eb989719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:49.789151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b5483-e9cb-481d-8227-dba6eb989719 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:49.789436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b5483-e9cb-481d-8227-dba6eb989719 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:49.789707 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b5483-e9cb-481d-8227-dba6eb989719 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:49.802036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346e5b2-7428-4e0a-a82e-1d6bde00df55 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:49.811845 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b5483-e9cb-481d-8227-dba6eb989719 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:49.813335 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b5483-e9cb-481d-8227-dba6eb989719 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:49.838839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e45dd-4a06-42a3-b1a7-11acbfc9f921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:49.840050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e45dd-4a06-42a3-b1a7-11acbfc9f921 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:49.840584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e45dd-4a06-42a3-b1a7-11acbfc9f921 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:49.841083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e45dd-4a06-42a3-b1a7-11acbfc9f921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-d53b5483-e9cb-481d-8227-dba6eb989719 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:49.843460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:49.844395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:49.845351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:49.845351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:49.845351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:49.845351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:49.845351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:49.845351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:49.845351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:49.845351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:49.845351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 21:38:49.845351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:49.851136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 309/618] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:38:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:49.857341 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b5483-e9cb-481d-8227-dba6eb989719 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:49.861485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e45dd-4a06-42a3-b1a7-11acbfc9f921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:49.863252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e45dd-4a06-42a3-b1a7-11acbfc9f921 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-de3e45dd-4a06-42a3-b1a7-11acbfc9f921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:49.883838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:49.884739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:49.885611 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:49.885611 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:49.885611 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:49.885611 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:49.888424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 310/619] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:49.894883 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e45dd-4a06-42a3-b1a7-11acbfc9f921 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:50.030859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e45dd-4a06-42a3-b1a7-11acbfc9f921 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:50.067789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade4c64-a14a-4833-abc7-2fca249e7fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:50.068464 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade4c64-a14a-4833-abc7-2fca249e7fb6 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:50.068721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade4c64-a14a-4833-abc7-2fca249e7fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:50.069926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade4c64-a14a-4833-abc7-2fca249e7fb6 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:50.070606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade4c64-a14a-4833-abc7-2fca249e7fb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:50.070948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade4c64-a14a-4833-abc7-2fca249e7fb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:50.072340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade4c64-a14a-4833-abc7-2fca249e7fb6 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:50.086607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b5483-e9cb-481d-8227-dba6eb989719 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:50.099082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 311/620] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:50.105113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade4c64-a14a-4833-abc7-2fca249e7fb6 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:50.139634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e8e72-d814-427d-82bb-2a574139f4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:50.140748 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e8e72-d814-427d-82bb-2a574139f4de None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:50.142381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e8e72-d814-427d-82bb-2a574139f4de None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:50.143620 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e8e72-d814-427d-82bb-2a574139f4de None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:50.147176 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e8e72-d814-427d-82bb-2a574139f4de None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:50.147694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e8e72-d814-427d-82bb-2a574139f4de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:50.149328 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e8e72-d814-427d-82bb-2a574139f4de None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:50.187189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 310/621] 10.222.0.35 () {56 vars in 1135 bytes} [Fri May 13 21:38:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:50.190562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e8e72-d814-427d-82bb-2a574139f4de None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:50.306338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade4c64-a14a-4833-abc7-2fca249e7fb6 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:50.316428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e8e72-d814-427d-82bb-2a574139f4de None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:50.351290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05804095-7f4e-44fd-a96c-f9ee08490dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:50.352077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05804095-7f4e-44fd-a96c-f9ee08490dcc None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:50.352343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05804095-7f4e-44fd-a96c-f9ee08490dcc None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:50.352617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05804095-7f4e-44fd-a96c-f9ee08490dcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:50.363031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05804095-7f4e-44fd-a96c-f9ee08490dcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:50.365214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05804095-7f4e-44fd-a96c-f9ee08490dcc None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-05804095-7f4e-44fd-a96c-f9ee08490dcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:50.383218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:50.384194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:50.385092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:50.385092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:50.385092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:50.385092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:50.385092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 311/622] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:50.391104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05804095-7f4e-44fd-a96c-f9ee08490dcc None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:50.392251 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ed2f8-9078-4e8d-b409-110adaeee3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:50.393446 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ed2f8-9078-4e8d-b409-110adaeee3a9 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:50.397145 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ed2f8-9078-4e8d-b409-110adaeee3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:50.398589 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ed2f8-9078-4e8d-b409-110adaeee3a9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:50.421104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ed2f8-9078-4e8d-b409-110adaeee3a9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:50.426740 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ed2f8-9078-4e8d-b409-110adaeee3a9 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-741ed2f8-9078-4e8d-b409-110adaeee3a9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:50.437609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:50.439199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:50.440804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:50.440804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:50.440804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:50.440804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:50.440804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:50.440804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:50.440804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:50.440804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:50.440804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 13 21:38:50.440804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:50.440804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 312/623] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:50.446490 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ed2f8-9078-4e8d-b409-110adaeee3a9 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:50.496347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05804095-7f4e-44fd-a96c-f9ee08490dcc None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:50.531633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcfd09-1d66-48de-b94f-e913a622944f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:50.532298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcfd09-1d66-48de-b94f-e913a622944f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:50.532577 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcfd09-1d66-48de-b94f-e913a622944f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:50.532849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcfd09-1d66-48de-b94f-e913a622944f None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:50.533561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcfd09-1d66-48de-b94f-e913a622944f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:50.533861 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcfd09-1d66-48de-b94f-e913a622944f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:50.535248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcfd09-1d66-48de-b94f-e913a622944f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:50.547270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 312/624] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:50.552581 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcfd09-1d66-48de-b94f-e913a622944f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:50.652598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcfd09-1d66-48de-b94f-e913a622944f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:50.686105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a748031-e0dc-48ff-9977-b47d2464a472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:50.686933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a748031-e0dc-48ff-9977-b47d2464a472 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:50.687212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a748031-e0dc-48ff-9977-b47d2464a472 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:50.687487 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a748031-e0dc-48ff-9977-b47d2464a472 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:50.690289 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ed2f8-9078-4e8d-b409-110adaeee3a9 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:50.694126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a748031-e0dc-48ff-9977-b47d2464a472 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:50.695583 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a748031-e0dc-48ff-9977-b47d2464a472 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-1a748031-e0dc-48ff-9977-b47d2464a472 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:50.710106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:50.712232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:50.714361 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:50.714361 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:50.714361 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:50.714361 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:50.714361 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:50.714361 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:50.714361 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:50.714361 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:50.714361 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 313/625] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:50.718735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a748031-e0dc-48ff-9977-b47d2464a472 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:50.791013 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42b916-9114-4ffb-af1e-848359719b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:50.791692 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42b916-9114-4ffb-af1e-848359719b12 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:50.791994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42b916-9114-4ffb-af1e-848359719b12 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:50.792270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42b916-9114-4ffb-af1e-848359719b12 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:50.793116 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42b916-9114-4ffb-af1e-848359719b12 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:50.793422 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42b916-9114-4ffb-af1e-848359719b12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:50.794888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42b916-9114-4ffb-af1e-848359719b12 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:50.817047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a748031-e0dc-48ff-9977-b47d2464a472 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:50.817632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 313/626] 10.222.0.35 () {56 vars in 1137 bytes} [Fri May 13 21:38:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:50.825342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42b916-9114-4ffb-af1e-848359719b12 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:50.876288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6f1ad-be1c-4db5-bf15-a1b3e05896b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:50.876954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f1ad-be1c-4db5-bf15-a1b3e05896b5 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:50.877233 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f1ad-be1c-4db5-bf15-a1b3e05896b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:50.877758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f1ad-be1c-4db5-bf15-a1b3e05896b5 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:50.879323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6f1ad-be1c-4db5-bf15-a1b3e05896b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:50.879636 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6f1ad-be1c-4db5-bf15-a1b3e05896b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:50.881071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6f1ad-be1c-4db5-bf15-a1b3e05896b5 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:50.905638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 314/627] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:50.927218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6f1ad-be1c-4db5-bf15-a1b3e05896b5 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:51.012436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42b916-9114-4ffb-af1e-848359719b12 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:51.077747 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6f1ad-be1c-4db5-bf15-a1b3e05896b5 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:51.078318 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931cabe-f2d9-4aaf-98a9-e6ef7b7077c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:51.078995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931cabe-f2d9-4aaf-98a9-e6ef7b7077c8 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:51.079317 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931cabe-f2d9-4aaf-98a9-e6ef7b7077c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:51.079587 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931cabe-f2d9-4aaf-98a9-e6ef7b7077c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:51.148208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931cabe-f2d9-4aaf-98a9-e6ef7b7077c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:51.149803 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931cabe-f2d9-4aaf-98a9-e6ef7b7077c8 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:51.152915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937f189-2178-44b5-9098-45ccf183389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:51.153370 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937f189-2178-44b5-9098-45ccf183389b None admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:51.153611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937f189-2178-44b5-9098-45ccf183389b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:51.153885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937f189-2178-44b5-9098-45ccf183389b None admin] PATH_INFO: `/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/4f9e07ed0a084c82b946aaf585de16fd/roles/deb38f497be14e78ae21e8dceca3772f` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:51.176568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937f189-2178-44b5-9098-45ccf183389b None admin] RBAC: Authorizing `identity:create_grant(project_id=ac834e84b2c84261ade311a6dfbbeb20, user_id=4f9e07ed0a084c82b946aaf585de16fd, role_id=deb38f497be14e78ae21e8dceca3772f)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:51.178036 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937f189-2178-44b5-9098-45ccf183389b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:51.192260 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 315/628] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:38:50 2022] PUT /identity/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/4f9e07ed0a084c82b946aaf585de16fd/roles/deb38f497be14e78ae21e8dceca3772f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:38:51.194598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8a12e-09e5-4dc8-aea7-e2ff30471463 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:51.194909 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8a12e-09e5-4dc8-aea7-e2ff30471463 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:51.195185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8a12e-09e5-4dc8-aea7-e2ff30471463 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:51.196660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 316/629] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:51.202720 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a20c69-e47c-4929-8929-26959f76ab97 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:51.203033 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a20c69-e47c-4929-8929-26959f76ab97 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:51.203297 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a20c69-e47c-4929-8929-26959f76ab97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:51.259774 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-a5a20c69-e47c-4929-8929-26959f76ab97 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:51.261046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a5a20c69-e47c-4929-8929-26959f76ab97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:51.288532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 314/630] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:51.290375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d89d8-69af-4f32-bf09-f8180d331853 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:51.291675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d89d8-69af-4f32-bf09-f8180d331853 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:51.292157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d89d8-69af-4f32-bf09-f8180d331853 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:51.298892 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 315/631] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:51.300001 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d89d8-69af-4f32-bf09-f8180d331853 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:51.358896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 317/632] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:51.361211 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada5fb-5b65-4595-99e7-d55e81f7f427 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:51.361660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada5fb-5b65-4595-99e7-d55e81f7f427 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:51.362257 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada5fb-5b65-4595-99e7-d55e81f7f427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:51.383810 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-88ada5fb-5b65-4595-99e7-d55e81f7f427 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:51.386241 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-88ada5fb-5b65-4595-99e7-d55e81f7f427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:51.418978 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d89d8-69af-4f32-bf09-f8180d331853 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:51.457262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799e348-6847-42a4-90a4-756ae30819d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:51.458107 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799e348-6847-42a4-90a4-756ae30819d0 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:51.458384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799e348-6847-42a4-90a4-756ae30819d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:51.458721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799e348-6847-42a4-90a4-756ae30819d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:51.470222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799e348-6847-42a4-90a4-756ae30819d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:51.471706 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799e348-6847-42a4-90a4-756ae30819d0 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:51.475648 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 318/633] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:51.476644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ada5fb-5b65-4595-99e7-d55e81f7f427 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-2799e348-6847-42a4-90a4-756ae30819d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:51.503311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:51.504255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:51.505154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:51.505154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:51.505154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:51.505154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:51.505154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 316/634] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:51.505439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799e348-6847-42a4-90a4-756ae30819d0 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:51.593423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ada5fb-5b65-4595-99e7-d55e81f7f427 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:51.622019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799e348-6847-42a4-90a4-756ae30819d0 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:51.626379 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8475e-3fcf-440f-98cb-56201b083214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:51.627201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8475e-3fcf-440f-98cb-56201b083214 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:51.627746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8475e-3fcf-440f-98cb-56201b083214 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:51.628017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8475e-3fcf-440f-98cb-56201b083214 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:51.639475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8475e-3fcf-440f-98cb-56201b083214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-8ec8475e-3fcf-440f-98cb-56201b083214 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:51.646955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:51.647922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:51.648760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:51.648760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:51.648760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:51.648760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:51.648760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:51.648760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:51.648760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:51.648760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:51.648760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:51.648760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:51.648760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:51.648760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:51.648760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 319/635] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:51.650068 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8475e-3fcf-440f-98cb-56201b083214 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:51.665130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d97ba2-e0d9-4b2b-83d1-772495ae035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:51.665922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d97ba2-e0d9-4b2b-83d1-772495ae035f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:51.666204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d97ba2-e0d9-4b2b-83d1-772495ae035f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:51.666536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d97ba2-e0d9-4b2b-83d1-772495ae035f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:51.676754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d97ba2-e0d9-4b2b-83d1-772495ae035f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-51d97ba2-e0d9-4b2b-83d1-772495ae035f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:51.684380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:51.685335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:51.686204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:51.686204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:51.686204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:51.686204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:51.686204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:51.686204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:51.686204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:51.686204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:51.686204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:51.686204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:51.686204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:51.686204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:51.690362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 317/636] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:51.691453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2da81-cfea-4a57-9318-7d30479dd1de None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:51.691744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2da81-cfea-4a57-9318-7d30479dd1de None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:51.692021 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2da81-cfea-4a57-9318-7d30479dd1de None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:51.697606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 318/637] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:51.697606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2da81-cfea-4a57-9318-7d30479dd1de None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:51.750048 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8475e-3fcf-440f-98cb-56201b083214 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:51.782721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd06b5c-6245-43fc-8924-e8ed852aeddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:51.783385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd06b5c-6245-43fc-8924-e8ed852aeddc None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:51.783664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd06b5c-6245-43fc-8924-e8ed852aeddc None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:51.783939 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd06b5c-6245-43fc-8924-e8ed852aeddc None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:51.784648 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd06b5c-6245-43fc-8924-e8ed852aeddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:51.784965 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd06b5c-6245-43fc-8924-e8ed852aeddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:51.786314 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd06b5c-6245-43fc-8924-e8ed852aeddc None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:51.793098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2da81-cfea-4a57-9318-7d30479dd1de None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:51.802415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 320/638] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:51.803300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd06b5c-6245-43fc-8924-e8ed852aeddc None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:51.826161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213abe7-4bba-4ce4-9e65-35e7a41ce86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:51.827384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213abe7-4bba-4ce4-9e65-35e7a41ce86b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:51.827664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213abe7-4bba-4ce4-9e65-35e7a41ce86b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:51.827943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213abe7-4bba-4ce4-9e65-35e7a41ce86b None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:51.828678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213abe7-4bba-4ce4-9e65-35e7a41ce86b None admin] RBAC: Adding query filter params (name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:51.828986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213abe7-4bba-4ce4-9e65-35e7a41ce86b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:51.830367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213abe7-4bba-4ce4-9e65-35e7a41ce86b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:51.838243 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 319/639] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:51.839546 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb745a-5062-4d3f-a2d1-7090239f3bfe None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:51.839898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb745a-5062-4d3f-a2d1-7090239f3bfe None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:51.840238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb745a-5062-4d3f-a2d1-7090239f3bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:51.856424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-32fb745a-5062-4d3f-a2d1-7090239f3bfe None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:51.857618 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-32fb745a-5062-4d3f-a2d1-7090239f3bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:51.910214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd06b5c-6245-43fc-8924-e8ed852aeddc None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:51.925881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 320/640] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:51.926746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb745a-5062-4d3f-a2d1-7090239f3bfe None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:51.957756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521864b-f6fc-4c49-af88-929fe188a8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:51.958431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521864b-f6fc-4c49-af88-929fe188a8a7 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:51.958732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521864b-f6fc-4c49-af88-929fe188a8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:51.959027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521864b-f6fc-4c49-af88-929fe188a8a7 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:51.959776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521864b-f6fc-4c49-af88-929fe188a8a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:51.960084 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521864b-f6fc-4c49-af88-929fe188a8a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:51.961488 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521864b-f6fc-4c49-af88-929fe188a8a7 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:51.969721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 321/641] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:51.970918 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521864b-f6fc-4c49-af88-929fe188a8a7 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:52.030945 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb745a-5062-4d3f-a2d1-7090239f3bfe None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:52.065980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47e32c-9a54-48e4-8777-610657ce3c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:52.067117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47e32c-9a54-48e4-8777-610657ce3c44 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:52.067596 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47e32c-9a54-48e4-8777-610657ce3c44 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:52.068067 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47e32c-9a54-48e4-8777-610657ce3c44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:52.071871 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521864b-f6fc-4c49-af88-929fe188a8a7 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:52.075522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47e32c-9a54-48e4-8777-610657ce3c44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:52.077844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47e32c-9a54-48e4-8777-610657ce3c44 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-1c47e32c-9a54-48e4-8777-610657ce3c44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:52.091453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:52.092973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:52.094054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:52.094054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:52.094054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:52.094054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:52.094054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:52.094054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:52.094054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:52.094054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:52.096145 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 321/642] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:52.097157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47e32c-9a54-48e4-8777-610657ce3c44 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:52.108420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc2a85-55e2-496e-b3a0-0f258ba82aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:52.109397 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc2a85-55e2-496e-b3a0-0f258ba82aa5 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:52.109678 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc2a85-55e2-496e-b3a0-0f258ba82aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:52.109953 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc2a85-55e2-496e-b3a0-0f258ba82aa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:52.120265 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc2a85-55e2-496e-b3a0-0f258ba82aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:52.121703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc2a85-55e2-496e-b3a0-0f258ba82aa5 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-ecbc2a85-55e2-496e-b3a0-0f258ba82aa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:52.138474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:52.140035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:52.146036 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:52.146036 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:52.146036 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:52.146036 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:52.146036 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 322/643] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:52.146036 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc2a85-55e2-496e-b3a0-0f258ba82aa5 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:52.202491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47e32c-9a54-48e4-8777-610657ce3c44 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:52.235227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc2a85-55e2-496e-b3a0-0f258ba82aa5 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:52.235818 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe77ff-8195-4ea3-a107-32643bafc96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:52.236721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe77ff-8195-4ea3-a107-32643bafc96d None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:52.237006 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe77ff-8195-4ea3-a107-32643bafc96d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:52.237287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe77ff-8195-4ea3-a107-32643bafc96d None admin] PATH_INFO: `/v3/roles/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:52.246943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe77ff-8195-4ea3-a107-32643bafc96d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-7bbe77ff-8195-4ea3-a107-32643bafc96d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:52.255305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:52.256221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:52.257102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:52.257102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:52.257102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:52.257102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:52.257102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:52.257102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:52.257102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:52.257102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:52.257102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:52.257102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:52.257102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:52.257102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:52.257102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 322/644] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:52.257812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe77ff-8195-4ea3-a107-32643bafc96d None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:52.270524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe6067-83a4-450d-9730-6d1115a855d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:52.271344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe6067-83a4-450d-9730-6d1115a855d2 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:52.271624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe6067-83a4-450d-9730-6d1115a855d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:52.271895 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe6067-83a4-450d-9730-6d1115a855d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:52.282337 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe6067-83a4-450d-9730-6d1115a855d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:52.283816 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe6067-83a4-450d-9730-6d1115a855d2 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-0abe6067-83a4-450d-9730-6d1115a855d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:52.300931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:52.301837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:52.302879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:52.302879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:52.302879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:52.302879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:52.302879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 323/645] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:52.303194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe6067-83a4-450d-9730-6d1115a855d2 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:52.365561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe77ff-8195-4ea3-a107-32643bafc96d None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:52.399004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12801d8d-848e-47a6-a3c4-0128efe91e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:52.399756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12801d8d-848e-47a6-a3c4-0128efe91e49 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:52.400125 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12801d8d-848e-47a6-a3c4-0128efe91e49 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:52.400368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12801d8d-848e-47a6-a3c4-0128efe91e49 None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:52.401077 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12801d8d-848e-47a6-a3c4-0128efe91e49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:52.401383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12801d8d-848e-47a6-a3c4-0128efe91e49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:52.402147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe6067-83a4-450d-9730-6d1115a855d2 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:52.402837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12801d8d-848e-47a6-a3c4-0128efe91e49 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:52.414832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 323/646] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:52.415651 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12801d8d-848e-47a6-a3c4-0128efe91e49 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:52.438561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7a0ce-0b7e-40c1-9648-59a815e79e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:52.439258 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7a0ce-0b7e-40c1-9648-59a815e79e3e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:52.439539 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7a0ce-0b7e-40c1-9648-59a815e79e3e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:52.439814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7a0ce-0b7e-40c1-9648-59a815e79e3e None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:52.440534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7a0ce-0b7e-40c1-9648-59a815e79e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:52.440837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7a0ce-0b7e-40c1-9648-59a815e79e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:52.442197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7a0ce-0b7e-40c1-9648-59a815e79e3e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:52.458298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 324/647] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:52.458298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7a0ce-0b7e-40c1-9648-59a815e79e3e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:52.518725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12801d8d-848e-47a6-a3c4-0128efe91e49 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:52.551215 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7a0ce-0b7e-40c1-9648-59a815e79e3e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:52.555279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14e2e0-2b1d-404f-9adf-df918e81ce4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:52.555942 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14e2e0-2b1d-404f-9adf-df918e81ce4f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:52.556222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14e2e0-2b1d-404f-9adf-df918e81ce4f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:52.556495 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14e2e0-2b1d-404f-9adf-df918e81ce4f None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:52.557226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14e2e0-2b1d-404f-9adf-df918e81ce4f None admin] RBAC: Adding query filter params (name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:52.557529 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14e2e0-2b1d-404f-9adf-df918e81ce4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:52.559322 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14e2e0-2b1d-404f-9adf-df918e81ce4f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:52.570876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 324/648] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:52.571819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14e2e0-2b1d-404f-9adf-df918e81ce4f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:52.590193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efb011-7c0a-47e2-b21d-a46da90b7efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:52.591002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efb011-7c0a-47e2-b21d-a46da90b7efd None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:52.591284 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efb011-7c0a-47e2-b21d-a46da90b7efd None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:52.591557 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efb011-7c0a-47e2-b21d-a46da90b7efd None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:52.592804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efb011-7c0a-47e2-b21d-a46da90b7efd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:52.593109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efb011-7c0a-47e2-b21d-a46da90b7efd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:52.594473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efb011-7c0a-47e2-b21d-a46da90b7efd None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:52.610335 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 325/649] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:52.610335 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efb011-7c0a-47e2-b21d-a46da90b7efd None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:52.679147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14e2e0-2b1d-404f-9adf-df918e81ce4f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:52.705672 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efb011-7c0a-47e2-b21d-a46da90b7efd None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:52.710482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1708dde-250c-4478-a2c5-db13f773de3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:52.711160 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1708dde-250c-4478-a2c5-db13f773de3d None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:52.711441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1708dde-250c-4478-a2c5-db13f773de3d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:52.711710 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1708dde-250c-4478-a2c5-db13f773de3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:52.712526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1708dde-250c-4478-a2c5-db13f773de3d None admin] RBAC: Adding query filter params (role.id=deb38f497be14e78ae21e8dceca3772f, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=86c4a81ce97b4fd78889fae6c36bf6cb) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:52.712833 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1708dde-250c-4478-a2c5-db13f773de3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:52.714254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1708dde-250c-4478-a2c5-db13f773de3d None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:52.720608 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 325/650] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:52 2022] GET /identity/v3/role_assignments?user.id=86c4a81ce97b4fd78889fae6c36bf6cb&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=deb38f497be14e78ae21e8dceca3772f => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:52.721366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1708dde-250c-4478-a2c5-db13f773de3d None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:52.759090 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff44243-e10e-4b50-a225-25f13da74e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:52.760349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff44243-e10e-4b50-a225-25f13da74e0a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:52.760819 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff44243-e10e-4b50-a225-25f13da74e0a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:52.761441 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff44243-e10e-4b50-a225-25f13da74e0a None admin] PATH_INFO: `/v3/users/glance` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:52.774489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff44243-e10e-4b50-a225-25f13da74e0a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:52.776293 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff44243-e10e-4b50-a225-25f13da74e0a None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-bff44243-e10e-4b50-a225-25f13da74e0a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:52.792061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:52.793075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:52.794015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:52.794015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:52.794015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:52.794015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:52.794015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:52.794015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:52.794015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:52.794015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:52.794015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 13 21:38:52.794015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:52.801384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 326/651] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:52.802636 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff44243-e10e-4b50-a225-25f13da74e0a None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:52.833594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1708dde-250c-4478-a2c5-db13f773de3d None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:52.869109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151cdd1-c68e-4385-8c7d-45410ea4630f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:52.869907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151cdd1-c68e-4385-8c7d-45410ea4630f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:52.870185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151cdd1-c68e-4385-8c7d-45410ea4630f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:52.870698 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151cdd1-c68e-4385-8c7d-45410ea4630f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:52.883312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151cdd1-c68e-4385-8c7d-45410ea4630f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:52.884800 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151cdd1-c68e-4385-8c7d-45410ea4630f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-9151cdd1-c68e-4385-8c7d-45410ea4630f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:52.903996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:52.904985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:52.905902 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:52.905902 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:52.905902 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:52.905902 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:52.905902 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 326/652] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:52.907180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151cdd1-c68e-4385-8c7d-45410ea4630f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:52.916264 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff44243-e10e-4b50-a225-25f13da74e0a None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:52.957545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca152c9-6ade-4989-a346-ba85d46d9efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:52.958349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca152c9-6ade-4989-a346-ba85d46d9efa None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:52.958630 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca152c9-6ade-4989-a346-ba85d46d9efa None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:52.958929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca152c9-6ade-4989-a346-ba85d46d9efa None admin] PATH_INFO: `/v3/users/nova` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:52.969007 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca152c9-6ade-4989-a346-ba85d46d9efa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:52.970461 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca152c9-6ade-4989-a346-ba85d46d9efa None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-2ca152c9-6ade-4989-a346-ba85d46d9efa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:52.988920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:52.989894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:52.991066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:52.991066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:52.991066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:52.991066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:52.991066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:52.991066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:52.991066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:52.991066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:52.991066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 21:38:52.991066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:52.991066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 327/653] 10.222.0.35 () {56 vars in 1093 bytes} [Fri May 13 21:38:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:52.995780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca152c9-6ade-4989-a346-ba85d46d9efa None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:53.033666 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151cdd1-c68e-4385-8c7d-45410ea4630f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:53.068187 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33736808-e77f-4623-8f20-08d209143fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:53.068899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33736808-e77f-4623-8f20-08d209143fea None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:53.069185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33736808-e77f-4623-8f20-08d209143fea None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:53.069453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33736808-e77f-4623-8f20-08d209143fea None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:53.070292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33736808-e77f-4623-8f20-08d209143fea None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:53.070601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33736808-e77f-4623-8f20-08d209143fea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:53.072060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33736808-e77f-4623-8f20-08d209143fea None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:53.089913 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 327/654] 10.222.0.35 () {56 vars in 1137 bytes} [Fri May 13 21:38:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:53.094825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33736808-e77f-4623-8f20-08d209143fea None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:53.123887 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca152c9-6ade-4989-a346-ba85d46d9efa None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:53.168899 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbae4f7-a4d6-4dc0-a915-f3c1ca324850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:53.170541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbae4f7-a4d6-4dc0-a915-f3c1ca324850 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:53.171803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbae4f7-a4d6-4dc0-a915-f3c1ca324850 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:53.172241 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbae4f7-a4d6-4dc0-a915-f3c1ca324850 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:53.173176 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbae4f7-a4d6-4dc0-a915-f3c1ca324850 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:53.173654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbae4f7-a4d6-4dc0-a915-f3c1ca324850 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:53.175285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbae4f7-a4d6-4dc0-a915-f3c1ca324850 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:53.191981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 328/655] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:53.193427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbae4f7-a4d6-4dc0-a915-f3c1ca324850 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:53.205540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33736808-e77f-4623-8f20-08d209143fea None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:53.239482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a50c3-d50d-4c9e-9308-796de92d9a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:53.240237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a50c3-d50d-4c9e-9308-796de92d9a82 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:53.240622 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a50c3-d50d-4c9e-9308-796de92d9a82 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:53.240967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a50c3-d50d-4c9e-9308-796de92d9a82 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:53.242032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a50c3-d50d-4c9e-9308-796de92d9a82 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:53.242159 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a50c3-d50d-4c9e-9308-796de92d9a82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:53.243701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a50c3-d50d-4c9e-9308-796de92d9a82 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:53.267038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 328/656] 10.222.0.35 () {56 vars in 1133 bytes} [Fri May 13 21:38:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:53.268143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a50c3-d50d-4c9e-9308-796de92d9a82 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:53.318181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbae4f7-a4d6-4dc0-a915-f3c1ca324850 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:53.356370 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c30d4-bc3e-4cd6-91d8-2aad1ee9f20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:53.357341 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c30d4-bc3e-4cd6-91d8-2aad1ee9f20f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:53.358393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c30d4-bc3e-4cd6-91d8-2aad1ee9f20f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:53.358888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c30d4-bc3e-4cd6-91d8-2aad1ee9f20f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:53.393135 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a50c3-d50d-4c9e-9308-796de92d9a82 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:53.427324 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c30d4-bc3e-4cd6-91d8-2aad1ee9f20f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:53.429124 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c30d4-bc3e-4cd6-91d8-2aad1ee9f20f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:53.435861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d488f-50cd-4ad3-96db-a984b4e418de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:53.436658 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d488f-50cd-4ad3-96db-a984b4e418de None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:53.436946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d488f-50cd-4ad3-96db-a984b4e418de None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:53.437222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d488f-50cd-4ad3-96db-a984b4e418de None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:53.446565 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d488f-50cd-4ad3-96db-a984b4e418de None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:53.448574 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d488f-50cd-4ad3-96db-a984b4e418de None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-d01d488f-50cd-4ad3-96db-a984b4e418de None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:53.465700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:53.466693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:53.468230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:53.468230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:53.468230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:53.468230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:53.468230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:53.468230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:53.468230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:53.468230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:53.468230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 13 21:38:53.468230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:53.468230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 329/657] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:53.468851 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d488f-50cd-4ad3-96db-a984b4e418de None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:53.538615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 329/658] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:53.538615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fcd95-bbc2-4c28-a10a-eb40ececa8f6 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:53.538615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fcd95-bbc2-4c28-a10a-eb40ececa8f6 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:53.538615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fcd95-bbc2-4c28-a10a-eb40ececa8f6 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:53.542944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 330/659] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:53.542944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fcd95-bbc2-4c28-a10a-eb40ececa8f6 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:53.568273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d488f-50cd-4ad3-96db-a984b4e418de None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:53.606825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4042334-9f71-403a-9f86-607623018dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:53.609153 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4042334-9f71-403a-9f86-607623018dd1 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:53.609753 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4042334-9f71-403a-9f86-607623018dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:53.610205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4042334-9f71-403a-9f86-607623018dd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:53.649011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fcd95-bbc2-4c28-a10a-eb40ececa8f6 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:53.675430 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4042334-9f71-403a-9f86-607623018dd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:53.676902 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4042334-9f71-403a-9f86-607623018dd1 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:53.683825 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0afbf-0ada-4bef-8233-96b84c26bd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:53.684493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0afbf-0ada-4bef-8233-96b84c26bd0a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:53.684773 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0afbf-0ada-4bef-8233-96b84c26bd0a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:53.685047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0afbf-0ada-4bef-8233-96b84c26bd0a None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:53.685931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0afbf-0ada-4bef-8233-96b84c26bd0a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:53.686238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0afbf-0ada-4bef-8233-96b84c26bd0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:53.687723 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0afbf-0ada-4bef-8233-96b84c26bd0a None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:53.706746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 331/660] 10.222.0.35 () {56 vars in 1139 bytes} [Fri May 13 21:38:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:53.706746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0afbf-0ada-4bef-8233-96b84c26bd0a None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:53.779181 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 330/661] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:53.781127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a2c43-4547-44c5-8107-f9bad0242f4d None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:53.781419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a2c43-4547-44c5-8107-f9bad0242f4d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:53.781698 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a2c43-4547-44c5-8107-f9bad0242f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:53.797311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-574a2c43-4547-44c5-8107-f9bad0242f4d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:53.798488 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-574a2c43-4547-44c5-8107-f9bad0242f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:53.802287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0afbf-0ada-4bef-8233-96b84c26bd0a None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:53.839664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d8990-0e1e-4cb3-b593-178a4607993f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:53.840425 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d8990-0e1e-4cb3-b593-178a4607993f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:53.840709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d8990-0e1e-4cb3-b593-178a4607993f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:53.840979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d8990-0e1e-4cb3-b593-178a4607993f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:53.851533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d8990-0e1e-4cb3-b593-178a4607993f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:53.852979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d8990-0e1e-4cb3-b593-178a4607993f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:53.862232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 331/662] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:53.863254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a2c43-4547-44c5-8107-f9bad0242f4d None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-7d6d8990-0e1e-4cb3-b593-178a4607993f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:53.878689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:53.879804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:53.880751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:53.880751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:53.880751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:53.880751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:53.882688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 332/663] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:53.883666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d8990-0e1e-4cb3-b593-178a4607993f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:53.967841 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a2c43-4547-44c5-8107-f9bad0242f4d None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:53.982396 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d8990-0e1e-4cb3-b593-178a4607993f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:54.006047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b3485-67df-40ba-9669-55973adb9c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:54.006693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b3485-67df-40ba-9669-55973adb9c15 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:54.007073 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b3485-67df-40ba-9669-55973adb9c15 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:54.007495 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b3485-67df-40ba-9669-55973adb9c15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:54.016433 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56745058-fc04-4e86-853c-9d7e00e74399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:54.017215 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56745058-fc04-4e86-853c-9d7e00e74399 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:54.017493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56745058-fc04-4e86-853c-9d7e00e74399 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:54.017762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56745058-fc04-4e86-853c-9d7e00e74399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:54.029204 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56745058-fc04-4e86-853c-9d7e00e74399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:54.031280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56745058-fc04-4e86-853c-9d7e00e74399 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-56745058-fc04-4e86-853c-9d7e00e74399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:54.048602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:54.049499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:54.050980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:54.050980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:54.050980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:54.050980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:54.050980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 333/664] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:54.050980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56745058-fc04-4e86-853c-9d7e00e74399 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:54.080375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b3485-67df-40ba-9669-55973adb9c15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:54.081826 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b3485-67df-40ba-9669-55973adb9c15 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:54.147953 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56745058-fc04-4e86-853c-9d7e00e74399 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:54.184965 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4486a5b-9724-4fdf-9fb9-2c97c0e90375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:54.185211 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 332/665] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:54.186415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b3485-67df-40ba-9669-55973adb9c15 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:54.187348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4486a5b-9724-4fdf-9fb9-2c97c0e90375 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:54.187841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4486a5b-9724-4fdf-9fb9-2c97c0e90375 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:54.188383 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4486a5b-9724-4fdf-9fb9-2c97c0e90375 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:54.189403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4486a5b-9724-4fdf-9fb9-2c97c0e90375 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:54.189834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4486a5b-9724-4fdf-9fb9-2c97c0e90375 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:54.191312 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4486a5b-9724-4fdf-9fb9-2c97c0e90375 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:54.207045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 334/666] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:54.207977 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4486a5b-9724-4fdf-9fb9-2c97c0e90375 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:54.285613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b3485-67df-40ba-9669-55973adb9c15 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:54.303373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4486a5b-9724-4fdf-9fb9-2c97c0e90375 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:54.319778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633730c-f0b6-4ec9-b542-bdb70908d63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:54.320555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633730c-f0b6-4ec9-b542-bdb70908d63f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:54.320839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633730c-f0b6-4ec9-b542-bdb70908d63f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:54.321111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633730c-f0b6-4ec9-b542-bdb70908d63f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:54.331342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633730c-f0b6-4ec9-b542-bdb70908d63f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:54.335613 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ca1b9-d733-452d-b2de-4ca26e91e6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:54.336265 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca1b9-d733-452d-b2de-4ca26e91e6ab None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:54.336546 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca1b9-d733-452d-b2de-4ca26e91e6ab None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:54.336821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca1b9-d733-452d-b2de-4ca26e91e6ab None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:54.337530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ca1b9-d733-452d-b2de-4ca26e91e6ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:54.337837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ca1b9-d733-452d-b2de-4ca26e91e6ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-3633730c-f0b6-4ec9-b542-bdb70908d63f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:54.339205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:54.340128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:54.341556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:54.341556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:54.341556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:54.341556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:54.341556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:54.341556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:54.341556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:54.341556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:54.341556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:54.341556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:54.341556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:38:54.341556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:54.342210 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ca1b9-d733-452d-b2de-4ca26e91e6ab None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:54.342282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 333/667] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:38:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:54.342282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633730c-f0b6-4ec9-b542-bdb70908d63f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:54.355988 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 335/668] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:54.357137 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ca1b9-d733-452d-b2de-4ca26e91e6ab None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:54.445759 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633730c-f0b6-4ec9-b542-bdb70908d63f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:54.456290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ca1b9-d733-452d-b2de-4ca26e91e6ab None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:54.478005 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8aaf5-b8de-4691-92fc-774a089bd7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:54.479038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8aaf5-b8de-4691-92fc-774a089bd7ba None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:54.479496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8aaf5-b8de-4691-92fc-774a089bd7ba None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:54.479935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8aaf5-b8de-4691-92fc-774a089bd7ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:54.489322 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abb585-b30d-4a01-ad67-bd384142780e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:54.490315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abb585-b30d-4a01-ad67-bd384142780e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:54.490605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abb585-b30d-4a01-ad67-bd384142780e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:54.490923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abb585-b30d-4a01-ad67-bd384142780e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:54.491671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8aaf5-b8de-4691-92fc-774a089bd7ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:54.493293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8aaf5-b8de-4691-92fc-774a089bd7ba None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:54.497596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abb585-b30d-4a01-ad67-bd384142780e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:54.499668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abb585-b30d-4a01-ad67-bd384142780e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-33e8aaf5-b8de-4691-92fc-774a089bd7ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:54.511528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:54.512584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:54.513839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:54.513839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:54.513839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:54.513839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-22abb585-b30d-4a01-ad67-bd384142780e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:54.514112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:54.515125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:54.516203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:54.516203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:54.516203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:54.516203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:54.516203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:54.516203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:54.516203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:54.516203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:54.522057 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 336/669] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:54.522057 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abb585-b30d-4a01-ad67-bd384142780e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:54.522517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 334/670] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:54.523521 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8aaf5-b8de-4691-92fc-774a089bd7ba None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:54.630938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abb585-b30d-4a01-ad67-bd384142780e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:54.631143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8aaf5-b8de-4691-92fc-774a089bd7ba None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:54.663591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb72664-3c77-442b-8c59-98c588adcf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:54.663885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70982b77-b36e-482d-8ade-4883887bc386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:54.664374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb72664-3c77-442b-8c59-98c588adcf86 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:54.664656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb72664-3c77-442b-8c59-98c588adcf86 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:54.664816 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70982b77-b36e-482d-8ade-4883887bc386 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:54.664931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb72664-3c77-442b-8c59-98c588adcf86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:54.665271 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70982b77-b36e-482d-8ade-4883887bc386 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:54.665651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70982b77-b36e-482d-8ade-4883887bc386 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:54.666398 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70982b77-b36e-482d-8ade-4883887bc386 None admin] RBAC: Adding query filter params (name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:54.666737 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70982b77-b36e-482d-8ade-4883887bc386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:54.668105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70982b77-b36e-482d-8ade-4883887bc386 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:54.672375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb72664-3c77-442b-8c59-98c588adcf86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:54.673801 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb72664-3c77-442b-8c59-98c588adcf86 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:54.681280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 337/671] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:38:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:54.681280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70982b77-b36e-482d-8ade-4883887bc386 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-8eb72664-3c77-442b-8c59-98c588adcf86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:54.688268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:54.689150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:54.690630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:54.690630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:54.690630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:54.690630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:54.690630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:54.690630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:54.690630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:54.690630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:54.690630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 335/672] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:54.690630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb72664-3c77-442b-8c59-98c588adcf86 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:54.782795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70982b77-b36e-482d-8ade-4883887bc386 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:54.789603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb72664-3c77-442b-8c59-98c588adcf86 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:54.815808 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a9c41-2ab9-40c9-bd70-d08e0d19820e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:54.816466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a9c41-2ab9-40c9-bd70-d08e0d19820e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:54.816814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a9c41-2ab9-40c9-bd70-d08e0d19820e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:54.817019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a9c41-2ab9-40c9-bd70-d08e0d19820e None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:54.817798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a9c41-2ab9-40c9-bd70-d08e0d19820e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:54.818105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a9c41-2ab9-40c9-bd70-d08e0d19820e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:54.819591 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a9c41-2ab9-40c9-bd70-d08e0d19820e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:54.827694 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9479-e1eb-4eb4-8791-627b61b30e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:54.828359 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9479-e1eb-4eb4-8791-627b61b30e70 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:54.828641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9479-e1eb-4eb4-8791-627b61b30e70 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:54.828920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9479-e1eb-4eb4-8791-627b61b30e70 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:54.829635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb9479-e1eb-4eb4-8791-627b61b30e70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:54.829940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb9479-e1eb-4eb4-8791-627b61b30e70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:54.836442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb9479-e1eb-4eb4-8791-627b61b30e70 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:54.836579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 338/673] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:54.836579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a9c41-2ab9-40c9-bd70-d08e0d19820e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:54.843854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 336/674] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:54.845385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9479-e1eb-4eb4-8791-627b61b30e70 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:54.940413 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a9c41-2ab9-40c9-bd70-d08e0d19820e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:54.950307 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9479-e1eb-4eb4-8791-627b61b30e70 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:54.975452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1dbd6-3895-4276-8ac0-419e387c5d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:54.976242 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1dbd6-3895-4276-8ac0-419e387c5d98 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:54.976524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1dbd6-3895-4276-8ac0-419e387c5d98 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:54.976802 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1dbd6-3895-4276-8ac0-419e387c5d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:54.983335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fbb00-e809-4f4a-bb6b-4f24ea520fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:54.983978 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fbb00-e809-4f4a-bb6b-4f24ea520fb7 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:54.984256 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fbb00-e809-4f4a-bb6b-4f24ea520fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:54.984530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fbb00-e809-4f4a-bb6b-4f24ea520fb7 None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:54.985298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27fbb00-e809-4f4a-bb6b-4f24ea520fb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:54.985603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27fbb00-e809-4f4a-bb6b-4f24ea520fb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:54.986955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1dbd6-3895-4276-8ac0-419e387c5d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:54.987144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27fbb00-e809-4f4a-bb6b-4f24ea520fb7 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:54.988380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1dbd6-3895-4276-8ac0-419e387c5d98 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:55.004141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 337/675] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:55.004141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fbb00-e809-4f4a-bb6b-4f24ea520fb7 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-f2c1dbd6-3895-4276-8ac0-419e387c5d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:55.006181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:55.007089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:55.007963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:55.007963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:55.007963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:55.007963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:55.007963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 339/676] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:55.008247 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1dbd6-3895-4276-8ac0-419e387c5d98 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:55.110434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fbb00-e809-4f4a-bb6b-4f24ea520fb7 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:55.113134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1dbd6-3895-4276-8ac0-419e387c5d98 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:55.144155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce7f55-cdf8-4559-8f41-d4fc84989fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:55.145093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce7f55-cdf8-4559-8f41-d4fc84989fac None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:55.145577 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce7f55-cdf8-4559-8f41-d4fc84989fac None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:55.145743 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90a2e4-05f9-44db-8648-8f8f6a5390a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:55.146082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce7f55-cdf8-4559-8f41-d4fc84989fac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:55.146537 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90a2e4-05f9-44db-8648-8f8f6a5390a1 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:55.146912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90a2e4-05f9-44db-8648-8f8f6a5390a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:55.147184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90a2e4-05f9-44db-8648-8f8f6a5390a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:55.147273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce7f55-cdf8-4559-8f41-d4fc84989fac None admin] RBAC: Adding query filter params (role.id=deb38f497be14e78ae21e8dceca3772f, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=cad57b046f3a43978d1c98bcc1a76d9d) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:55.147563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce7f55-cdf8-4559-8f41-d4fc84989fac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:55.149514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce7f55-cdf8-4559-8f41-d4fc84989fac None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:55.154653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90a2e4-05f9-44db-8648-8f8f6a5390a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:55.156336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90a2e4-05f9-44db-8648-8f8f6a5390a1 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:55.160665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 338/677] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:54 2022] GET /identity/v3/role_assignments?user.id=cad57b046f3a43978d1c98bcc1a76d9d&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=deb38f497be14e78ae21e8dceca3772f => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:55.161491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce7f55-cdf8-4559-8f41-d4fc84989fac None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-da90a2e4-05f9-44db-8648-8f8f6a5390a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:55.183948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:55.185059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:55.187144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:55.187144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:55.187144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:55.187144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:55.187144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:55.187144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:55.187144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:55.187144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:55.187144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 340/678] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:55.187662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90a2e4-05f9-44db-8648-8f8f6a5390a1 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:55.273523 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce7f55-cdf8-4559-8f41-d4fc84989fac None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:55.296930 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90a2e4-05f9-44db-8648-8f8f6a5390a1 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:55.309884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f143bec-8132-4605-a28a-c71c3c580dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:55.311117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f143bec-8132-4605-a28a-c71c3c580dd3 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:55.311543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f143bec-8132-4605-a28a-c71c3c580dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:55.311714 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f143bec-8132-4605-a28a-c71c3c580dd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:55.312518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f143bec-8132-4605-a28a-c71c3c580dd3 None admin] RBAC: Adding query filter params (role.id=deb38f497be14e78ae21e8dceca3772f, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=316ef43c6c304bd3a27e085bf4da8a5e) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:55.312830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f143bec-8132-4605-a28a-c71c3c580dd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:55.314301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f143bec-8132-4605-a28a-c71c3c580dd3 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:55.320528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 339/679] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:55 2022] GET /identity/v3/role_assignments?user.id=316ef43c6c304bd3a27e085bf4da8a5e&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=deb38f497be14e78ae21e8dceca3772f => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:55.320528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0b584-2212-4d0e-b82d-fb7c05e40963 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:55.320528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0b584-2212-4d0e-b82d-fb7c05e40963 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:55.320528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0b584-2212-4d0e-b82d-fb7c05e40963 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:55.320528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 340/680] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:55.320528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0b584-2212-4d0e-b82d-fb7c05e40963 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:55.351958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d18d28-483e-41f8-8b70-33722a0e3389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:55.352647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d18d28-483e-41f8-8b70-33722a0e3389 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:55.352928 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d18d28-483e-41f8-8b70-33722a0e3389 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:55.353203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d18d28-483e-41f8-8b70-33722a0e3389 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:55.353941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d18d28-483e-41f8-8b70-33722a0e3389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:55.354247 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d18d28-483e-41f8-8b70-33722a0e3389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:55.356824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d18d28-483e-41f8-8b70-33722a0e3389 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:55.377674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 341/681] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:55.379098 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913396d-ff62-402c-ad27-cd20aabcfd01 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:55.379392 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913396d-ff62-402c-ad27-cd20aabcfd01 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:55.379665 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913396d-ff62-402c-ad27-cd20aabcfd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:55.397174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-d913396d-ff62-402c-ad27-cd20aabcfd01 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:55.398507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d913396d-ff62-402c-ad27-cd20aabcfd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:55.477245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0b584-2212-4d0e-b82d-fb7c05e40963 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:55.496204 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 342/682] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:55.497106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913396d-ff62-402c-ad27-cd20aabcfd01 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:55.527128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe224cd-59e8-43a4-bc4b-28e88ae9fe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:55.527861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe224cd-59e8-43a4-bc4b-28e88ae9fe34 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:55.528150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe224cd-59e8-43a4-bc4b-28e88ae9fe34 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:55.530509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe224cd-59e8-43a4-bc4b-28e88ae9fe34 None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:55.530509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe224cd-59e8-43a4-bc4b-28e88ae9fe34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:55.530509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe224cd-59e8-43a4-bc4b-28e88ae9fe34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:55.534726 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe224cd-59e8-43a4-bc4b-28e88ae9fe34 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:55.544282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 341/683] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:55.545746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe224cd-59e8-43a4-bc4b-28e88ae9fe34 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:55.624598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913396d-ff62-402c-ad27-cd20aabcfd01 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:55.663664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165adcf-df05-410f-ad51-637c6e6f3586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:55.664380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165adcf-df05-410f-ad51-637c6e6f3586 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:55.664661 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165adcf-df05-410f-ad51-637c6e6f3586 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:55.664936 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165adcf-df05-410f-ad51-637c6e6f3586 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:55.668218 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe224cd-59e8-43a4-bc4b-28e88ae9fe34 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:55.674349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165adcf-df05-410f-ad51-637c6e6f3586 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:55.676927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165adcf-df05-410f-ad51-637c6e6f3586 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-e165adcf-df05-410f-ad51-637c6e6f3586 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:55.700189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:55.701238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:55.702162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:55.702162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:55.702162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:55.702162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:55.702162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:55.702162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:55.702162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:55.702162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:55.702162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 21:38:55.702162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:55.704246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 343/684] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:38:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:55.708829 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165adcf-df05-410f-ad51-637c6e6f3586 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:55.712746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebe3fe-1542-44ce-aaf8-031ead934c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:55.713677 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebe3fe-1542-44ce-aaf8-031ead934c01 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:55.713962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebe3fe-1542-44ce-aaf8-031ead934c01 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:55.714235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebe3fe-1542-44ce-aaf8-031ead934c01 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:55.714915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebe3fe-1542-44ce-aaf8-031ead934c01 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:55.716295 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebe3fe-1542-44ce-aaf8-031ead934c01 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-51ebe3fe-1542-44ce-aaf8-031ead934c01 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:55.730881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:55.731837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:55.733274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 342/685] 10.222.0.35 () {56 vars in 1129 bytes} [Fri May 13 21:38:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:55.734839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebe3fe-1542-44ce-aaf8-031ead934c01 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:55.815034 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165adcf-df05-410f-ad51-637c6e6f3586 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:55.838331 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebe3fe-1542-44ce-aaf8-031ead934c01 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:55.850661 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29867414-cd90-4d7b-b72f-5a6c4cb4cedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:55.851737 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29867414-cd90-4d7b-b72f-5a6c4cb4cedb None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:55.852290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29867414-cd90-4d7b-b72f-5a6c4cb4cedb None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:55.852760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29867414-cd90-4d7b-b72f-5a6c4cb4cedb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:55.853789 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29867414-cd90-4d7b-b72f-5a6c4cb4cedb None admin] RBAC: Adding query filter params (role.id=deb38f497be14e78ae21e8dceca3772f, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=c4df058606c448a1bb757020e869ac33) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:55.854315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29867414-cd90-4d7b-b72f-5a6c4cb4cedb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:55.856007 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29867414-cd90-4d7b-b72f-5a6c4cb4cedb None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:55.868865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 344/686] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:55 2022] GET /identity/v3/role_assignments?user.id=c4df058606c448a1bb757020e869ac33&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=deb38f497be14e78ae21e8dceca3772f => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:55.870145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29867414-cd90-4d7b-b72f-5a6c4cb4cedb None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:55.874425 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3d6f2-f2c9-4c1d-9437-ae34f23adae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:55.875142 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3d6f2-f2c9-4c1d-9437-ae34f23adae2 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:55.875428 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3d6f2-f2c9-4c1d-9437-ae34f23adae2 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:55.875703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3d6f2-f2c9-4c1d-9437-ae34f23adae2 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:55.876546 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3d6f2-f2c9-4c1d-9437-ae34f23adae2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:55.876856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3d6f2-f2c9-4c1d-9437-ae34f23adae2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:55.878629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3d6f2-f2c9-4c1d-9437-ae34f23adae2 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:55.898603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 343/687] 10.222.0.35 () {56 vars in 1135 bytes} [Fri May 13 21:38:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:55.908162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3d6f2-f2c9-4c1d-9437-ae34f23adae2 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:55.990453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29867414-cd90-4d7b-b72f-5a6c4cb4cedb None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:56.028582 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b60885-0835-48bd-be05-43e2cff3719c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:56.030442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b60885-0835-48bd-be05-43e2cff3719c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:56.030958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b60885-0835-48bd-be05-43e2cff3719c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:56.031573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b60885-0835-48bd-be05-43e2cff3719c None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:56.032559 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b60885-0835-48bd-be05-43e2cff3719c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:56.033405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b60885-0835-48bd-be05-43e2cff3719c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:56.035089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b60885-0835-48bd-be05-43e2cff3719c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:56.042151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 345/688] 10.222.0.35 () {56 vars in 1124 bytes} [Fri May 13 21:38:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:56.047124 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b60885-0835-48bd-be05-43e2cff3719c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:56.134021 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3d6f2-f2c9-4c1d-9437-ae34f23adae2 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:56.161706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b60885-0835-48bd-be05-43e2cff3719c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:56.192159 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4f9fe-c82b-49cc-a7c6-a10d9db83505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:56.192919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4f9fe-c82b-49cc-a7c6-a10d9db83505 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:56.193207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4f9fe-c82b-49cc-a7c6-a10d9db83505 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:56.193485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4f9fe-c82b-49cc-a7c6-a10d9db83505 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:56.198587 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b24fea-3df4-4f6d-850a-ee434c522da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:56.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b24fea-3df4-4f6d-850a-ee434c522da6 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:56.200024 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b24fea-3df4-4f6d-850a-ee434c522da6 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:56.200507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b24fea-3df4-4f6d-850a-ee434c522da6 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:56.201980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b24fea-3df4-4f6d-850a-ee434c522da6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:56.202482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b24fea-3df4-4f6d-850a-ee434c522da6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:56.204078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b24fea-3df4-4f6d-850a-ee434c522da6 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:56.210122 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 346/689] 10.222.0.35 () {56 vars in 1124 bytes} [Fri May 13 21:38:56 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:56.270881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4f9fe-c82b-49cc-a7c6-a10d9db83505 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:56.272717 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4f9fe-c82b-49cc-a7c6-a10d9db83505 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:56.380994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 344/690] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 21:38:56.386883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b24fea-3df4-4f6d-850a-ee434c522da6 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:56.491107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b24fea-3df4-4f6d-850a-ee434c522da6 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:56.532661 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4e13b-b78e-4d83-9424-642bb3f6bfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:56.533804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e13b-b78e-4d83-9424-642bb3f6bfb0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:56.534283 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e13b-b78e-4d83-9424-642bb3f6bfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:56.534791 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e13b-b78e-4d83-9424-642bb3f6bfb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:56.548608 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4e13b-b78e-4d83-9424-642bb3f6bfb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:56.550064 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4e13b-b78e-4d83-9424-642bb3f6bfb0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-e6d4e13b-b78e-4d83-9424-642bb3f6bfb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:56.571445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:56.572349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:56.573235 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:56.573235 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:56.573235 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:56.573235 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:56.573932 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 347/691] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:56.578624 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4f9fe-c82b-49cc-a7c6-a10d9db83505 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:56.725034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4f9fe-c82b-49cc-a7c6-a10d9db83505 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:56.772029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d1431-5d90-4b51-8342-4db427bc8b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:56.772706 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d1431-5d90-4b51-8342-4db427bc8b1f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:56.772990 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d1431-5d90-4b51-8342-4db427bc8b1f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:56.773264 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d1431-5d90-4b51-8342-4db427bc8b1f None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:56.773999 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d1431-5d90-4b51-8342-4db427bc8b1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:56.774299 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d1431-5d90-4b51-8342-4db427bc8b1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:56.775707 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d1431-5d90-4b51-8342-4db427bc8b1f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:56.793478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 345/692] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:56.798932 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4e13b-b78e-4d83-9424-642bb3f6bfb0 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:56.913938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4e13b-b78e-4d83-9424-642bb3f6bfb0 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:56.954818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ee2bc-3b87-4138-9fc0-b5a3812b07c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:56.955619 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ee2bc-3b87-4138-9fc0-b5a3812b07c9 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:56.955904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ee2bc-3b87-4138-9fc0-b5a3812b07c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:56.956180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ee2bc-3b87-4138-9fc0-b5a3812b07c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:56.967949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ee2bc-3b87-4138-9fc0-b5a3812b07c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:56.969546 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ee2bc-3b87-4138-9fc0-b5a3812b07c9 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-fc2ee2bc-3b87-4138-9fc0-b5a3812b07c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:38:56.984772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:56.985679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:56.986550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:38:56.986550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:38:56.986550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:38:56.986550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:38:56.986550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:38:56.986550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:38:56.986550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:38:56.986550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:56.986550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 348/693] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:56.991722 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d1431-5d90-4b51-8342-4db427bc8b1f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:57.121733 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d1431-5d90-4b51-8342-4db427bc8b1f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:57.166238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eea91d-0afc-4f3f-8786-be28b2aebfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:57.166932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eea91d-0afc-4f3f-8786-be28b2aebfae None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:57.167214 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eea91d-0afc-4f3f-8786-be28b2aebfae None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:57.167492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eea91d-0afc-4f3f-8786-be28b2aebfae None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:57.168289 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eea91d-0afc-4f3f-8786-be28b2aebfae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:57.168602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eea91d-0afc-4f3f-8786-be28b2aebfae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:57.170673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eea91d-0afc-4f3f-8786-be28b2aebfae None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:57.183612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 346/694] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:38:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:57.191915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ee2bc-3b87-4138-9fc0-b5a3812b07c9 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:57.342660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ee2bc-3b87-4138-9fc0-b5a3812b07c9 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:57.382259 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da4fd3-c882-4188-8088-4ee399731673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:57.382882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da4fd3-c882-4188-8088-4ee399731673 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:57.383167 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da4fd3-c882-4188-8088-4ee399731673 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:57.383445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da4fd3-c882-4188-8088-4ee399731673 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:57.384272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da4fd3-c882-4188-8088-4ee399731673 None admin] RBAC: Adding query filter params (role.id=deb38f497be14e78ae21e8dceca3772f, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=4f9e07ed0a084c82b946aaf585de16fd) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:57.384583 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da4fd3-c882-4188-8088-4ee399731673 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:57.386026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da4fd3-c882-4188-8088-4ee399731673 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:57.392176 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 349/695] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:38:57 2022] GET /identity/v3/role_assignments?user.id=4f9e07ed0a084c82b946aaf585de16fd&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=deb38f497be14e78ae21e8dceca3772f => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:57.907499 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07126ad-80ab-4030-bb89-d06683c22728 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:57.908043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07126ad-80ab-4030-bb89-d06683c22728 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:57.908043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07126ad-80ab-4030-bb89-d06683c22728 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:57.909619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 347/696] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:57.914925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cc5de-0ff4-43b6-b9f9-797fad431fc4 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:57.915220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cc5de-0ff4-43b6-b9f9-797fad431fc4 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:57.915498 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cc5de-0ff4-43b6-b9f9-797fad431fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:57.934810 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-e06cc5de-0ff4-43b6-b9f9-797fad431fc4 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:57.936549 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-e06cc5de-0ff4-43b6-b9f9-797fad431fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:58.006659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 350/697] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:58.031780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07126ad-80ab-4030-bb89-d06683c22728 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:58.040676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95edf36-f752-426d-b798-b767c345e11b None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:58.041229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95edf36-f752-426d-b798-b767c345e11b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:58.041701 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95edf36-f752-426d-b798-b767c345e11b None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:58.043291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 351/698] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:58.050063 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b3e2b-8ea7-4fe4-9de5-c8ea1ad5c206 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:58.050646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b3e2b-8ea7-4fe4-9de5-c8ea1ad5c206 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:58.051168 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b3e2b-8ea7-4fe4-9de5-c8ea1ad5c206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:58.071403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ae0b3e2b-8ea7-4fe4-9de5-c8ea1ad5c206 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:58.072571 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ae0b3e2b-8ea7-4fe4-9de5-c8ea1ad5c206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:58.147946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07126ad-80ab-4030-bb89-d06683c22728 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:58.165937 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 352/699] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:58.187492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb671e-1fb9-41f1-b6ef-8f3a8e96d479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:58.191076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb671e-1fb9-41f1-b6ef-8f3a8e96d479 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:58.191076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb671e-1fb9-41f1-b6ef-8f3a8e96d479 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:58.198731 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb671e-1fb9-41f1-b6ef-8f3a8e96d479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:58.198887 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b3e2b-8ea7-4fe4-9de5-c8ea1ad5c206 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:58.207256 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb671e-1fb9-41f1-b6ef-8f3a8e96d479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:58.208795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb671e-1fb9-41f1-b6ef-8f3a8e96d479 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-b5cb671e-1fb9-41f1-b6ef-8f3a8e96d479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:58.228534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:58.229479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:58.230358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:58.230358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:58.230358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:58.230358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:38:58.230358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 348/700] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:58.236803 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb671e-1fb9-41f1-b6ef-8f3a8e96d479 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:58.305959 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b3e2b-8ea7-4fe4-9de5-c8ea1ad5c206 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:58.346010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbcfe0-19e4-409d-92c5-31add9d30724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:58.346831 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbcfe0-19e4-409d-92c5-31add9d30724 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:58.347115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbcfe0-19e4-409d-92c5-31add9d30724 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:58.347387 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbcfe0-19e4-409d-92c5-31add9d30724 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:58.367234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbcfe0-19e4-409d-92c5-31add9d30724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-78fbcfe0-19e4-409d-92c5-31add9d30724 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:38:58.374916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:58.375818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:58.376646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:58.376646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:58.376646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:58.376646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:58.376646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:38:58.376646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:38:58.376646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:38:58.376646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:38:58.376646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:38:58.376646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:38:58.376646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:38:58.376646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:58.376646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 353/701] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:38:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:58.382187 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbcfe0-19e4-409d-92c5-31add9d30724 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:58.462388 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb671e-1fb9-41f1-b6ef-8f3a8e96d479 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:58.482892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbcfe0-19e4-409d-92c5-31add9d30724 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:58.518401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b56d74-cfed-44ef-a05b-2a9ac0443c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:58.518639 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa167169-a782-4d75-a6e0-ea7149ed1684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:58.519103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b56d74-cfed-44ef-a05b-2a9ac0443c57 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:58.519284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa167169-a782-4d75-a6e0-ea7149ed1684 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:58.519398 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b56d74-cfed-44ef-a05b-2a9ac0443c57 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:58.519568 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa167169-a782-4d75-a6e0-ea7149ed1684 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:58.519667 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b56d74-cfed-44ef-a05b-2a9ac0443c57 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:58.519848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa167169-a782-4d75-a6e0-ea7149ed1684 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:58.520370 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b56d74-cfed-44ef-a05b-2a9ac0443c57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:58.520556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa167169-a782-4d75-a6e0-ea7149ed1684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:58.520709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b56d74-cfed-44ef-a05b-2a9ac0443c57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:58.520858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa167169-a782-4d75-a6e0-ea7149ed1684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:58.522037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b56d74-cfed-44ef-a05b-2a9ac0443c57 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:58.522241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa167169-a782-4d75-a6e0-ea7149ed1684 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:58.531551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 354/702] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:38:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:58.536872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b56d74-cfed-44ef-a05b-2a9ac0443c57 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:58.547197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 349/703] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:58.552679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa167169-a782-4d75-a6e0-ea7149ed1684 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:58.641942 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b56d74-cfed-44ef-a05b-2a9ac0443c57 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:58.677212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98266832-a230-4df8-9b7e-142649200857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:58.678036 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98266832-a230-4df8-9b7e-142649200857 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:58.678317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98266832-a230-4df8-9b7e-142649200857 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:58.678592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98266832-a230-4df8-9b7e-142649200857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:58.679227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa167169-a782-4d75-a6e0-ea7149ed1684 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:58.690193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98266832-a230-4df8-9b7e-142649200857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:58.691691 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98266832-a230-4df8-9b7e-142649200857 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-98266832-a230-4df8-9b7e-142649200857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:38:58.709590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:58.710532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:58.711493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:38:58.711493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:38:58.711493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:38:58.711493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:58.711493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 355/704] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:38:58.713656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292b087-92fa-4789-af1d-4d2b061f57f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:58.714267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292b087-92fa-4789-af1d-4d2b061f57f2 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:58.714544 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292b087-92fa-4789-af1d-4d2b061f57f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:58.714841 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292b087-92fa-4789-af1d-4d2b061f57f2 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:58.715628 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292b087-92fa-4789-af1d-4d2b061f57f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:58.716533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98266832-a230-4df8-9b7e-142649200857 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:58.717193 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292b087-92fa-4789-af1d-4d2b061f57f2 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:58.754811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 350/705] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:38:58 2022] POST /identity/v3/users => generated 254 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:38:58.828496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98266832-a230-4df8-9b7e-142649200857 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:58.863533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17848af4-2504-4677-96b7-bb0344738647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:58.864200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17848af4-2504-4677-96b7-bb0344738647 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:58.864477 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17848af4-2504-4677-96b7-bb0344738647 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:58.864753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17848af4-2504-4677-96b7-bb0344738647 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:58.865477 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17848af4-2504-4677-96b7-bb0344738647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:58.865779 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17848af4-2504-4677-96b7-bb0344738647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:58.867234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17848af4-2504-4677-96b7-bb0344738647 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:58.876147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fde81-4ea5-4130-9356-911455cdc357 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:58.876474 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fde81-4ea5-4130-9356-911455cdc357 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:58.876700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fde81-4ea5-4130-9356-911455cdc357 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:58.878633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 351/706] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:38:58.881798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 356/707] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:58.884162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd0c2c-45a9-46fc-ab25-83a6d68e3527 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:58.884455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd0c2c-45a9-46fc-ab25-83a6d68e3527 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:58.884737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd0c2c-45a9-46fc-ab25-83a6d68e3527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:58.887092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17848af4-2504-4677-96b7-bb0344738647 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:58.905592 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-1abd0c2c-45a9-46fc-ab25-83a6d68e3527 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:58.907302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-1abd0c2c-45a9-46fc-ab25-83a6d68e3527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:58.996528 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17848af4-2504-4677-96b7-bb0344738647 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:59.027460 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 352/708] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:38:59.028841 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b243c-7404-4dba-8c25-d11bbe2c271b None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:59.029130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b243c-7404-4dba-8c25-d11bbe2c271b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:59.029415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b243c-7404-4dba-8c25-d11bbe2c271b None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:59.031366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 353/709] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:38:59.034357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eeaf93-0aa9-4624-aa28-9daa687e0f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:59.035189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeaf93-0aa9-4624-aa28-9daa687e0f62 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:59.035649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeaf93-0aa9-4624-aa28-9daa687e0f62 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:59.035922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeaf93-0aa9-4624-aa28-9daa687e0f62 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:59.046501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcd2e5-a641-4928-823f-789720fb8ee9 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:59.046801 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcd2e5-a641-4928-823f-789720fb8ee9 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:59.047078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcd2e5-a641-4928-823f-789720fb8ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:59.049031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eeaf93-0aa9-4624-aa28-9daa687e0f62 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:59.050683 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eeaf93-0aa9-4624-aa28-9daa687e0f62 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:59.065743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-4dbcd2e5-a641-4928-823f-789720fb8ee9 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:38:59.066981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-4dbcd2e5-a641-4928-823f-789720fb8ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-00eeaf93-0aa9-4624-aa28-9daa687e0f62 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:38:59.068961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:59.069864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:59.070875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:59.070875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:59.070875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:38:59.070875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:38:59.070875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:38:59.070875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:38:59.070875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:38:59.070875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:38:59.070875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 21:38:59.070875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:59.070875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 357/710] 10.222.0.35 () {56 vars in 1093 bytes} [Fri May 13 21:38:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:59.071463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eeaf93-0aa9-4624-aa28-9daa687e0f62 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:59.155176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 354/711] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:38:59.156102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcd2e5-a641-4928-823f-789720fb8ee9 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:59.193182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eeaf93-0aa9-4624-aa28-9daa687e0f62 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:59.236193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c254ab5-d3c4-46d6-b482-fbf63c5e0e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:59.236993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c254ab5-d3c4-46d6-b482-fbf63c5e0e93 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:59.237281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c254ab5-d3c4-46d6-b482-fbf63c5e0e93 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:59.237848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c254ab5-d3c4-46d6-b482-fbf63c5e0e93 None admin] PATH_INFO: `/v3/services/network` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:59.238812 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c254ab5-d3c4-46d6-b482-fbf63c5e0e93 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:59.240208 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c254ab5-d3c4-46d6-b482-fbf63c5e0e93 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-8c254ab5-d3c4-46d6-b482-fbf63c5e0e93 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:38:59.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:38:59.259182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:38:59.260105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 358/712] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:38:59 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:38:59.260875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c254ab5-d3c4-46d6-b482-fbf63c5e0e93 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:59.270885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcd2e5-a641-4928-823f-789720fb8ee9 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:59.304654 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b3a8d-fb40-40b4-8ef2-f6dffeac7b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:59.305270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b3a8d-fb40-40b4-8ef2-f6dffeac7b28 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:59.305602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b3a8d-fb40-40b4-8ef2-f6dffeac7b28 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:59.305890 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b3a8d-fb40-40b4-8ef2-f6dffeac7b28 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:59.306734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b3a8d-fb40-40b4-8ef2-f6dffeac7b28 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:59.307056 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b3a8d-fb40-40b4-8ef2-f6dffeac7b28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:59.308494 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b3a8d-fb40-40b4-8ef2-f6dffeac7b28 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:59.327480 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 355/713] 10.222.0.35 () {56 vars in 1133 bytes} [Fri May 13 21:38:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:59.328360 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b3a8d-fb40-40b4-8ef2-f6dffeac7b28 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:59.390436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c254ab5-d3c4-46d6-b482-fbf63c5e0e93 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:59.429789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b3a8d-fb40-40b4-8ef2-f6dffeac7b28 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:59.435541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc958a-b6b9-44ae-8964-2e292eacc2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:59.436400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc958a-b6b9-44ae-8964-2e292eacc2d9 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:59.436824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc958a-b6b9-44ae-8964-2e292eacc2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:59.437256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc958a-b6b9-44ae-8964-2e292eacc2d9 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:59.438540 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc958a-b6b9-44ae-8964-2e292eacc2d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:59.440488 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc958a-b6b9-44ae-8964-2e292eacc2d9 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:59.461217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 359/714] 10.222.0.35 () {60 vars in 1141 bytes} [Fri May 13 21:38:59 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:38:59.462924 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc958a-b6b9-44ae-8964-2e292eacc2d9 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:59.475483 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be838ef-db50-44cc-a8fa-d96519cffd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:59.476458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be838ef-db50-44cc-a8fa-d96519cffd81 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:59.476843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be838ef-db50-44cc-a8fa-d96519cffd81 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:59.477294 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be838ef-db50-44cc-a8fa-d96519cffd81 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:59.478202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be838ef-db50-44cc-a8fa-d96519cffd81 None admin] RBAC: Adding query filter params (name=network) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:59.479027 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be838ef-db50-44cc-a8fa-d96519cffd81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:59.480302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be838ef-db50-44cc-a8fa-d96519cffd81 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:59.486551 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 356/715] 10.222.0.35 () {56 vars in 1112 bytes} [Fri May 13 21:38:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:38:59.492481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be838ef-db50-44cc-a8fa-d96519cffd81 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:38:59.592547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc958a-b6b9-44ae-8964-2e292eacc2d9 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:59.631911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b47cd2-a3cd-4af2-890a-69d566bab6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:59.632813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b47cd2-a3cd-4af2-890a-69d566bab6ad None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:59.633240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b47cd2-a3cd-4af2-890a-69d566bab6ad None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:59.633664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b47cd2-a3cd-4af2-890a-69d566bab6ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:59.711857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be838ef-db50-44cc-a8fa-d96519cffd81 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:38:59.718415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b47cd2-a3cd-4af2-890a-69d566bab6ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:59.720098 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b47cd2-a3cd-4af2-890a-69d566bab6ad None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:59.748803 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5e937-1980-4a0e-9a8d-6a126c4cb3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:38:59.749465 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5e937-1980-4a0e-9a8d-6a126c4cb3b7 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:38:59.749741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5e937-1980-4a0e-9a8d-6a126c4cb3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:38:59.750013 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5e937-1980-4a0e-9a8d-6a126c4cb3b7 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:38:59.750738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5e937-1980-4a0e-9a8d-6a126c4cb3b7 None admin] RBAC: Adding query filter params (type=network) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:38:59.751050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5e937-1980-4a0e-9a8d-6a126c4cb3b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:38:59.752697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5e937-1980-4a0e-9a8d-6a126c4cb3b7 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:38:59.759564 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 357/716] 10.222.0.35 () {56 vars in 1112 bytes} [Fri May 13 21:38:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:38:59.960891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 360/717] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:38:59 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:38:59.966030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5e937-1980-4a0e-9a8d-6a126c4cb3b7 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:00.077279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5e937-1980-4a0e-9a8d-6a126c4cb3b7 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:00.114035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8ec9f-852a-43eb-846c-f9da19612ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:00.114857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8ec9f-852a-43eb-846c-f9da19612ba4 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:00.115144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8ec9f-852a-43eb-846c-f9da19612ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:00.115423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8ec9f-852a-43eb-846c-f9da19612ba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:00.126177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8ec9f-852a-43eb-846c-f9da19612ba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:00.127983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8ec9f-852a-43eb-846c-f9da19612ba4 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-c7d8ec9f-852a-43eb-846c-f9da19612ba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:00.145439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:00.146382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:00.147993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:00.147993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:00.147993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:00.147993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:00.147993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 358/718] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:00.156191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b47cd2-a3cd-4af2-890a-69d566bab6ad None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:00.314949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b47cd2-a3cd-4af2-890a-69d566bab6ad None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:00.364294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d693730-11fb-4e6b-b417-9bfc1e9ff6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:00.365325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d693730-11fb-4e6b-b417-9bfc1e9ff6df None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:00.365802 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d693730-11fb-4e6b-b417-9bfc1e9ff6df None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:00.366286 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d693730-11fb-4e6b-b417-9bfc1e9ff6df None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:00.367268 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d693730-11fb-4e6b-b417-9bfc1e9ff6df None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:00.367766 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d693730-11fb-4e6b-b417-9bfc1e9ff6df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:00.369333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d693730-11fb-4e6b-b417-9bfc1e9ff6df None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:00.385596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 361/719] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:00.391745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8ec9f-852a-43eb-846c-f9da19612ba4 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:00.517975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8ec9f-852a-43eb-846c-f9da19612ba4 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:00.565791 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fceed-f806-4dd4-8efb-0dbac61fde22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:00.566944 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fceed-f806-4dd4-8efb-0dbac61fde22 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:00.567431 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fceed-f806-4dd4-8efb-0dbac61fde22 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:00.567917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fceed-f806-4dd4-8efb-0dbac61fde22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:00.575340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fceed-f806-4dd4-8efb-0dbac61fde22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:00.577178 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fceed-f806-4dd4-8efb-0dbac61fde22 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-305fceed-f806-4dd4-8efb-0dbac61fde22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:00.596950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:00.597854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:00.603162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:00.603162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:00.603162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:00.603162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:00.603162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:00.603162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:00.603162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:00.603162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:00.604994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 359/720] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:00.611071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d693730-11fb-4e6b-b417-9bfc1e9ff6df None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:00.690560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555016cb-c019-4b4a-8ed8-da3d277cf281 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:00.691227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555016cb-c019-4b4a-8ed8-da3d277cf281 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:00.691697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555016cb-c019-4b4a-8ed8-da3d277cf281 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:00.693260 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 360/721] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:39:00.698224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae9312-79ae-467d-8157-77e5eb9fafc7 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:00.698509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae9312-79ae-467d-8157-77e5eb9fafc7 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:00.698815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae9312-79ae-467d-8157-77e5eb9fafc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:00.721703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-dfae9312-79ae-467d-8157-77e5eb9fafc7 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:00.723915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-dfae9312-79ae-467d-8157-77e5eb9fafc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:00.763758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d693730-11fb-4e6b-b417-9bfc1e9ff6df None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:00.829504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b334c-303d-436e-91d7-a580855f2bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:00.830089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b334c-303d-436e-91d7-a580855f2bd9 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:00.830369 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b334c-303d-436e-91d7-a580855f2bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:00.830641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b334c-303d-436e-91d7-a580855f2bd9 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:00.831462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b334c-303d-436e-91d7-a580855f2bd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:00.831769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b334c-303d-436e-91d7-a580855f2bd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:00.833346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b334c-303d-436e-91d7-a580855f2bd9 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:00.841016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 361/722] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:00 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 21:39:00.849972 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 362/723] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:00.855583 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae9312-79ae-467d-8157-77e5eb9fafc7 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:00.885647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b334c-303d-436e-91d7-a580855f2bd9 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:01.019009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b334c-303d-436e-91d7-a580855f2bd9 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:01.048809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae9312-79ae-467d-8157-77e5eb9fafc7 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:01.058652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccafc8-0eaf-4b1c-a84e-11212f376946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:01.059786 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccafc8-0eaf-4b1c-a84e-11212f376946 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:01.060299 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccafc8-0eaf-4b1c-a84e-11212f376946 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:01.060865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccafc8-0eaf-4b1c-a84e-11212f376946 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:01.071483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dccafc8-0eaf-4b1c-a84e-11212f376946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-8dccafc8-0eaf-4b1c-a84e-11212f376946 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:01.079382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:01.080344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:01.081177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:01.081177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:01.081177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:01.081177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:01.081177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:01.081177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:01.081177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:01.081177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:01.081177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:01.081177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:01.081177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:01.081177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:01.087853 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 363/724] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:39:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:01.090585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccafc8-0eaf-4b1c-a84e-11212f376946 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:01.119670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53195a1d-b27d-41ef-8278-c64edd6684cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:01.120389 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53195a1d-b27d-41ef-8278-c64edd6684cf None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:01.120612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53195a1d-b27d-41ef-8278-c64edd6684cf None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:01.120832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53195a1d-b27d-41ef-8278-c64edd6684cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:01.121695 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53195a1d-b27d-41ef-8278-c64edd6684cf None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=316ef43c6c304bd3a27e085bf4da8a5e) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:01.122007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53195a1d-b27d-41ef-8278-c64edd6684cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:01.124096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53195a1d-b27d-41ef-8278-c64edd6684cf None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:01.139281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 362/725] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:39:00 2022] GET /identity/v3/role_assignments?user.id=316ef43c6c304bd3a27e085bf4da8a5e&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:01.209049 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccafc8-0eaf-4b1c-a84e-11212f376946 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:01.246496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a15915-363e-4592-9b7a-0682399d4a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:01.247241 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a15915-363e-4592-9b7a-0682399d4a9e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:01.247521 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a15915-363e-4592-9b7a-0682399d4a9e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:01.248343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a15915-363e-4592-9b7a-0682399d4a9e None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:01.249101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a15915-363e-4592-9b7a-0682399d4a9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:01.249401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a15915-363e-4592-9b7a-0682399d4a9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:01.250805 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a15915-363e-4592-9b7a-0682399d4a9e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:01.265008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 364/726] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:39:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:01.271858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53195a1d-b27d-41ef-8278-c64edd6684cf None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:01.385640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53195a1d-b27d-41ef-8278-c64edd6684cf None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:01.426282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebea0b3-4822-4e0e-9d8a-2913b2e9c739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:01.427110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebea0b3-4822-4e0e-9d8a-2913b2e9c739 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:01.427385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebea0b3-4822-4e0e-9d8a-2913b2e9c739 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:01.427657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebea0b3-4822-4e0e-9d8a-2913b2e9c739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:01.438276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebea0b3-4822-4e0e-9d8a-2913b2e9c739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:01.439744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebea0b3-4822-4e0e-9d8a-2913b2e9c739 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-4ebea0b3-4822-4e0e-9d8a-2913b2e9c739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:01.468866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:01.469828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:01.470697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:01.470697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:01.470697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:01.470697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:01.473026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 363/727] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:01.478329 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a15915-363e-4592-9b7a-0682399d4a9e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:01.602304 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a15915-363e-4592-9b7a-0682399d4a9e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:01.650562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccd4bb-8b10-4844-bed8-274d6012caf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:01.651232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccd4bb-8b10-4844-bed8-274d6012caf0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:01.651513 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccd4bb-8b10-4844-bed8-274d6012caf0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:01.651702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccd4bb-8b10-4844-bed8-274d6012caf0 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:01.652430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccd4bb-8b10-4844-bed8-274d6012caf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:01.652736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccd4bb-8b10-4844-bed8-274d6012caf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:01.654144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccd4bb-8b10-4844-bed8-274d6012caf0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:01.669683 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 365/728] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:01.674973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebea0b3-4822-4e0e-9d8a-2913b2e9c739 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:01.814772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df971c-4270-468d-a692-9d68fce3dcac None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:01.815123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df971c-4270-468d-a692-9d68fce3dcac None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:01.815339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df971c-4270-468d-a692-9d68fce3dcac None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:01.816703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 366/729] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:01.825433 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16df30-40ec-4334-a43e-76cbf07c737f None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:01.825715 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16df30-40ec-4334-a43e-76cbf07c737f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:01.826030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16df30-40ec-4334-a43e-76cbf07c737f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:01.826230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebea0b3-4822-4e0e-9d8a-2913b2e9c739 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:01.859563 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5e16df30-40ec-4334-a43e-76cbf07c737f None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:01.860772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5e16df30-40ec-4334-a43e-76cbf07c737f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:01.870856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24233714-8548-47b0-8a27-98fa19bfe368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:01.871661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24233714-8548-47b0-8a27-98fa19bfe368 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:01.871942 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24233714-8548-47b0-8a27-98fa19bfe368 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:01.872218 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24233714-8548-47b0-8a27-98fa19bfe368 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:01.883189 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24233714-8548-47b0-8a27-98fa19bfe368 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:01.884672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24233714-8548-47b0-8a27-98fa19bfe368 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-24233714-8548-47b0-8a27-98fa19bfe368 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:01.900838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:01.901850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:01.902848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:01.902848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:01.902848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:01.902848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:01.902848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:01.902848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:01.902848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:01.902848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:01.902848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 21:39:01.902848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:01.902848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 364/730] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:39:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:01.908096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24233714-8548-47b0-8a27-98fa19bfe368 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:01.994869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 367/731] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:39:02.019573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16df30-40ec-4334-a43e-76cbf07c737f None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:02.089699 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24233714-8548-47b0-8a27-98fa19bfe368 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:02.160613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088fad1-8c8d-463a-8286-21eb711b989a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:02.161281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088fad1-8c8d-463a-8286-21eb711b989a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:02.161571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088fad1-8c8d-463a-8286-21eb711b989a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:02.161849 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088fad1-8c8d-463a-8286-21eb711b989a None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:02.162775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088fad1-8c8d-463a-8286-21eb711b989a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:02.163090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088fad1-8c8d-463a-8286-21eb711b989a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:02.165213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088fad1-8c8d-463a-8286-21eb711b989a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:02.180720 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16df30-40ec-4334-a43e-76cbf07c737f None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:02.202775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 365/732] 10.222.0.35 () {56 vars in 1135 bytes} [Fri May 13 21:39:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:02.208739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088fad1-8c8d-463a-8286-21eb711b989a None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:02.259927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e66fb-c33e-4641-bbe0-f2e16568d414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:02.260732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e66fb-c33e-4641-bbe0-f2e16568d414 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:02.261079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e66fb-c33e-4641-bbe0-f2e16568d414 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:02.261356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e66fb-c33e-4641-bbe0-f2e16568d414 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:02.287627 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e66fb-c33e-4641-bbe0-f2e16568d414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-6a4e66fb-c33e-4641-bbe0-f2e16568d414 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:02.295633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:02.296614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:02.297448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:02.297448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:02.297448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:02.297448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:02.297448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:02.297448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:02.297448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:02.297448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:02.297448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:02.297448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:02.297448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:39:02.297448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:02.307829 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 368/733] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:39:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:02.319761 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e66fb-c33e-4641-bbe0-f2e16568d414 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:02.411215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088fad1-8c8d-463a-8286-21eb711b989a None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:02.444696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f268a21-231b-49ea-8f27-0ae42ebfee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:02.445336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f268a21-231b-49ea-8f27-0ae42ebfee28 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:02.445623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f268a21-231b-49ea-8f27-0ae42ebfee28 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:02.445900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f268a21-231b-49ea-8f27-0ae42ebfee28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:02.456758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e66fb-c33e-4641-bbe0-f2e16568d414 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:02.492654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396a2b5-311e-4121-9e9b-2a224abfb79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:02.493309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396a2b5-311e-4121-9e9b-2a224abfb79f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:02.493591 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396a2b5-311e-4121-9e9b-2a224abfb79f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:02.493862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396a2b5-311e-4121-9e9b-2a224abfb79f None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:02.494599 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396a2b5-311e-4121-9e9b-2a224abfb79f None admin] RBAC: Adding query filter params (name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:02.495326 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396a2b5-311e-4121-9e9b-2a224abfb79f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:02.496980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396a2b5-311e-4121-9e9b-2a224abfb79f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:02.510881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f268a21-231b-49ea-8f27-0ae42ebfee28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:02.511799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 369/734] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:39:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:02.512589 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f268a21-231b-49ea-8f27-0ae42ebfee28 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:02.513147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20a25d-2a7b-4727-9754-bc781f2141d4 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:02.514000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20a25d-2a7b-4727-9754-bc781f2141d4 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:02.514285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20a25d-2a7b-4727-9754-bc781f2141d4 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:02.519715 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 370/735] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:02.519715 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396451a-feb0-4360-acf5-c85e710d18ec None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:02.519715 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396451a-feb0-4360-acf5-c85e710d18ec None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:02.519715 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396451a-feb0-4360-acf5-c85e710d18ec None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:02.519715 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 371/736] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:02.519715 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396451a-feb0-4360-acf5-c85e710d18ec None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:02.623802 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396451a-feb0-4360-acf5-c85e710d18ec None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:02.634974 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 366/737] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:39:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 21:39:02.634974 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d12235-2374-49ac-9a20-619452351301 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:02.634974 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d12235-2374-49ac-9a20-619452351301 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:02.634974 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d12235-2374-49ac-9a20-619452351301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:02.654745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a7d12235-2374-49ac-9a20-619452351301 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:02.656231 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a7d12235-2374-49ac-9a20-619452351301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:02.657507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f5543-2f6b-43b7-8855-348ed99f84bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:02.658300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f5543-2f6b-43b7-8855-348ed99f84bc None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:02.658588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f5543-2f6b-43b7-8855-348ed99f84bc None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:02.658995 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f5543-2f6b-43b7-8855-348ed99f84bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:02.670406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f5543-2f6b-43b7-8855-348ed99f84bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:02.671828 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f5543-2f6b-43b7-8855-348ed99f84bc None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-760f5543-2f6b-43b7-8855-348ed99f84bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:02.689129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:02.690058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:02.691849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:02.691849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:02.691849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:02.691849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:02.691849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 372/738] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:02.691849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bb8d5-a5bd-4da5-ab23-183a327f3b9c None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:02.691849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bb8d5-a5bd-4da5-ab23-183a327f3b9c None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:02.692243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bb8d5-a5bd-4da5-ab23-183a327f3b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:02.710496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-822bb8d5-a5bd-4da5-ab23-183a327f3b9c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:02.711724 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-822bb8d5-a5bd-4da5-ab23-183a327f3b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:02.735215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 367/739] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:39:02.736566 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d12235-2374-49ac-9a20-619452351301 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:02.793557 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 373/740] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 21:39:02.794491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822bb8d5-a5bd-4da5-ab23-183a327f3b9c None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:02.853978 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d12235-2374-49ac-9a20-619452351301 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:02.889914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822bb8d5-a5bd-4da5-ab23-183a327f3b9c None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:02.896286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cd41d-a66f-439c-a087-6466597d129c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:02.897458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cd41d-a66f-439c-a087-6466597d129c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:02.897879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cd41d-a66f-439c-a087-6466597d129c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:02.898423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cd41d-a66f-439c-a087-6466597d129c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:02.911341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cd41d-a66f-439c-a087-6466597d129c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:02.912807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cd41d-a66f-439c-a087-6466597d129c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:02.926668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a010d3-6653-4e2c-b59d-022914dac53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:02.927536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a010d3-6653-4e2c-b59d-022914dac53c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:02.927961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a010d3-6653-4e2c-b59d-022914dac53c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:02.928378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a010d3-6653-4e2c-b59d-022914dac53c None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:02.929226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a010d3-6653-4e2c-b59d-022914dac53c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:02.929670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a010d3-6653-4e2c-b59d-022914dac53c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:02.931198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a010d3-6653-4e2c-b59d-022914dac53c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-8b6cd41d-a66f-439c-a087-6466597d129c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:02.933155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:02.934167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:02.935118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:02.935118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:02.935118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:02.935118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:02.935118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 368/741] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:02.935118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cd41d-a66f-439c-a087-6466597d129c None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:02.951580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 374/742] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:02.952696 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a010d3-6653-4e2c-b59d-022914dac53c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:03.034618 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cd41d-a66f-439c-a087-6466597d129c None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:03.049717 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a010d3-6653-4e2c-b59d-022914dac53c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:03.071848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e7636-966c-4181-90d7-86025ad4db11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:03.073593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e7636-966c-4181-90d7-86025ad4db11 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:03.074083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e7636-966c-4181-90d7-86025ad4db11 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:03.074735 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e7636-966c-4181-90d7-86025ad4db11 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:03.075739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e7636-966c-4181-90d7-86025ad4db11 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:03.077585 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e7636-966c-4181-90d7-86025ad4db11 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:03.083300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436154d-01ba-448c-90b1-44beb2214465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:03.083963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436154d-01ba-448c-90b1-44beb2214465 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:03.084244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436154d-01ba-448c-90b1-44beb2214465 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:03.084521 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436154d-01ba-448c-90b1-44beb2214465 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:03.085237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436154d-01ba-448c-90b1-44beb2214465 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:03.086598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436154d-01ba-448c-90b1-44beb2214465 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-263e7636-966c-4181-90d7-86025ad4db11 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:03.090981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:03.093373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 21:39:03.094247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:03.097878 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 369/743] 10.222.0.35 () {56 vars in 1129 bytes} [Fri May 13 21:39:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:03.098725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e7636-966c-4181-90d7-86025ad4db11 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:03.111121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 375/744] 10.222.0.35 () {60 vars in 1141 bytes} [Fri May 13 21:39:02 2022] POST /identity/v3/services => generated 240 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:39:03.111121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436154d-01ba-448c-90b1-44beb2214465 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:03.229020 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e7636-966c-4181-90d7-86025ad4db11 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:03.258626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436154d-01ba-448c-90b1-44beb2214465 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:03.275380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b18031-4b04-48e5-9005-01fb2b651d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:03.276182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b18031-4b04-48e5-9005-01fb2b651d06 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:03.276524 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b18031-4b04-48e5-9005-01fb2b651d06 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:03.276838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b18031-4b04-48e5-9005-01fb2b651d06 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:03.277658 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b18031-4b04-48e5-9005-01fb2b651d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:03.278042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b18031-4b04-48e5-9005-01fb2b651d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:03.280308 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b18031-4b04-48e5-9005-01fb2b651d06 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:03.294230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 370/745] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:03.295871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b18031-4b04-48e5-9005-01fb2b651d06 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:03.300257 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056da28-2a4d-47e8-9a90-6df08eb4893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:03.301390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056da28-2a4d-47e8-9a90-6df08eb4893c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:03.301881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056da28-2a4d-47e8-9a90-6df08eb4893c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:03.302374 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056da28-2a4d-47e8-9a90-6df08eb4893c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:03.310650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056da28-2a4d-47e8-9a90-6df08eb4893c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:03.312443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056da28-2a4d-47e8-9a90-6df08eb4893c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-6056da28-2a4d-47e8-9a90-6df08eb4893c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:03.328220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:03.329223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:03.330178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:03.330178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:03.330178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:03.330178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:03.330178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:03.330178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:03.330178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:03.330178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:03.330178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 21:39:03.330178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:03.330178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 376/746] 10.222.0.35 () {56 vars in 1117 bytes} [Fri May 13 21:39:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:03.330178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056da28-2a4d-47e8-9a90-6df08eb4893c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:03.424095 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b18031-4b04-48e5-9005-01fb2b651d06 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:03.453618 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056da28-2a4d-47e8-9a90-6df08eb4893c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:03.468663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925469c-54c3-4d84-9da6-ba002383f645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:03.469496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925469c-54c3-4d84-9da6-ba002383f645 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:03.469781 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925469c-54c3-4d84-9da6-ba002383f645 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:03.470057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925469c-54c3-4d84-9da6-ba002383f645 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:03.470794 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925469c-54c3-4d84-9da6-ba002383f645 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:03.471180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925469c-54c3-4d84-9da6-ba002383f645 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:03.472483 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925469c-54c3-4d84-9da6-ba002383f645 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:03.477425 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 371/747] 10.222.0.35 () {56 vars in 1124 bytes} [Fri May 13 21:39:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:03.478209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925469c-54c3-4d84-9da6-ba002383f645 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:03.494938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77478a95-7b7a-4502-8365-cd79455ab167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:03.496076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77478a95-7b7a-4502-8365-cd79455ab167 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:03.496612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77478a95-7b7a-4502-8365-cd79455ab167 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:03.497091 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77478a95-7b7a-4502-8365-cd79455ab167 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:03.504848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77478a95-7b7a-4502-8365-cd79455ab167 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:03.506671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77478a95-7b7a-4502-8365-cd79455ab167 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-77478a95-7b7a-4502-8365-cd79455ab167 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:03.525443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:03.526400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:03.527317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:03.527317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:03.527317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:03.527317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:03.527317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:03.527317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:03.527317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:03.527317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:03.530454 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 377/748] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:03.531558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77478a95-7b7a-4502-8365-cd79455ab167 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:03.588808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925469c-54c3-4d84-9da6-ba002383f645 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:03.623719 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c8381-4254-42e7-bfcb-317150ce9c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:03.624245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c8381-4254-42e7-bfcb-317150ce9c1d None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:03.624531 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c8381-4254-42e7-bfcb-317150ce9c1d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:03.624814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c8381-4254-42e7-bfcb-317150ce9c1d None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:03.625653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c8381-4254-42e7-bfcb-317150ce9c1d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:03.625966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c8381-4254-42e7-bfcb-317150ce9c1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:03.627818 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c8381-4254-42e7-bfcb-317150ce9c1d None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:03.641202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77478a95-7b7a-4502-8365-cd79455ab167 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:03.650783 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 372/749] 10.222.0.35 () {56 vars in 1149 bytes} [Fri May 13 21:39:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:03.650783 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c8381-4254-42e7-bfcb-317150ce9c1d None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:03.688019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62324333-0870-4e3f-8fac-2974b0d58457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:03.688830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62324333-0870-4e3f-8fac-2974b0d58457 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:03.689213 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62324333-0870-4e3f-8fac-2974b0d58457 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:03.689494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62324333-0870-4e3f-8fac-2974b0d58457 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:03.690203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62324333-0870-4e3f-8fac-2974b0d58457 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:03.690509 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62324333-0870-4e3f-8fac-2974b0d58457 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:03.692064 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62324333-0870-4e3f-8fac-2974b0d58457 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:03.698054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 378/750] 10.222.0.35 () {56 vars in 1124 bytes} [Fri May 13 21:39:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:03.698884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62324333-0870-4e3f-8fac-2974b0d58457 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:03.765227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c8381-4254-42e7-bfcb-317150ce9c1d None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:03.798476 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed7403-7cde-47b1-adb8-58d45a28bb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:03.799287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed7403-7cde-47b1-adb8-58d45a28bb22 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:03.799668 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed7403-7cde-47b1-adb8-58d45a28bb22 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:03.800044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed7403-7cde-47b1-adb8-58d45a28bb22 None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:03.800730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62324333-0870-4e3f-8fac-2974b0d58457 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:03.801036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed7403-7cde-47b1-adb8-58d45a28bb22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:03.801443 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed7403-7cde-47b1-adb8-58d45a28bb22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:03.803063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed7403-7cde-47b1-adb8-58d45a28bb22 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:03.814951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 373/751] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:03.815892 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed7403-7cde-47b1-adb8-58d45a28bb22 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:03.838807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f051a-60ca-436a-a124-5d506ff25127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:03.839693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f051a-60ca-436a-a124-5d506ff25127 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:03.839970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f051a-60ca-436a-a124-5d506ff25127 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:03.840239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f051a-60ca-436a-a124-5d506ff25127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:03.905261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f051a-60ca-436a-a124-5d506ff25127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:03.906959 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f051a-60ca-436a-a124-5d506ff25127 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:03.912879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed7403-7cde-47b1-adb8-58d45a28bb22 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:03.945040 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18aab2-0c26-4dea-b43d-52cdfd64c4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:03.945689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18aab2-0c26-4dea-b43d-52cdfd64c4ed None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:03.945969 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18aab2-0c26-4dea-b43d-52cdfd64c4ed None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:03.946243 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18aab2-0c26-4dea-b43d-52cdfd64c4ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:03.947123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18aab2-0c26-4dea-b43d-52cdfd64c4ed None admin] RBAC: Adding query filter params (interface=public, service_id=64b8bf335eb04fb7a8b60f4384208362, region_id=RegionOne) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:03.947434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18aab2-0c26-4dea-b43d-52cdfd64c4ed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:03.948796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18aab2-0c26-4dea-b43d-52cdfd64c4ed None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:03.956946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 374/752] 10.222.0.35 () {56 vars in 1251 bytes} [Fri May 13 21:39:03 2022] GET /identity/v3/endpoints?service_id=64b8bf335eb04fb7a8b60f4384208362&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:03.958367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058021c-5796-47e7-88cd-b3cc66cba1ec None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:03.958658 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058021c-5796-47e7-88cd-b3cc66cba1ec None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:03.958965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058021c-5796-47e7-88cd-b3cc66cba1ec None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:03.960339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 375/753] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:03.960875 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058021c-5796-47e7-88cd-b3cc66cba1ec None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:04.020785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 379/754] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:39:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:39:04.021919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f051a-60ca-436a-a124-5d506ff25127 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:04.128583 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f051a-60ca-436a-a124-5d506ff25127 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:04.166324 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c23779-a31c-4a47-ba9e-4bc3e0c70abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:04.167095 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c23779-a31c-4a47-ba9e-4bc3e0c70abb None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:04.167382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c23779-a31c-4a47-ba9e-4bc3e0c70abb None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:04.167692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c23779-a31c-4a47-ba9e-4bc3e0c70abb None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:04.168372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c23779-a31c-4a47-ba9e-4bc3e0c70abb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:04.169769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c23779-a31c-4a47-ba9e-4bc3e0c70abb None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:04.175135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 380/755] 10.222.0.35 () {56 vars in 1087 bytes} [Fri May 13 21:39:04 2022] GET /identity/v3/services => generated 772 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:04.176363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe151ed-7b2c-47c7-a443-1c332fd4d809 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:04.176650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe151ed-7b2c-47c7-a443-1c332fd4d809 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:04.176926 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe151ed-7b2c-47c7-a443-1c332fd4d809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:04.197090 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-fbe151ed-7b2c-47c7-a443-1c332fd4d809 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:04.198328 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-fbe151ed-7b2c-47c7-a443-1c332fd4d809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:04.205768 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058021c-5796-47e7-88cd-b3cc66cba1ec None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:04.244914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dfd96e-4e56-4593-81ee-87f61ec7fe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:04.246460 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfd96e-4e56-4593-81ee-87f61ec7fe09 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:04.246986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfd96e-4e56-4593-81ee-87f61ec7fe09 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:04.247443 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfd96e-4e56-4593-81ee-87f61ec7fe09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:04.249122 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dfd96e-4e56-4593-81ee-87f61ec7fe09 None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=4f9e07ed0a084c82b946aaf585de16fd) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:04.249653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dfd96e-4e56-4593-81ee-87f61ec7fe09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:04.251379 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dfd96e-4e56-4593-81ee-87f61ec7fe09 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:04.267316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 376/756] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:39:03 2022] GET /identity/v3/role_assignments?user.id=4f9e07ed0a084c82b946aaf585de16fd&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 259 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:04.268587 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dfd96e-4e56-4593-81ee-87f61ec7fe09 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:04.273758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 381/757] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:04.324145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe151ed-7b2c-47c7-a443-1c332fd4d809 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:04.404384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dfd96e-4e56-4593-81ee-87f61ec7fe09 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:04.459640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999b932-3421-4f60-bfdd-1973636572c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:04.460812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999b932-3421-4f60-bfdd-1973636572c2 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:04.461292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999b932-3421-4f60-bfdd-1973636572c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:04.461807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999b932-3421-4f60-bfdd-1973636572c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:04.473919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999b932-3421-4f60-bfdd-1973636572c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:04.475785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999b932-3421-4f60-bfdd-1973636572c2 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:04.490865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe151ed-7b2c-47c7-a443-1c332fd4d809 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-d999b932-3421-4f60-bfdd-1973636572c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:04.494811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:04.495773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:04.496663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:04.496663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:04.496663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:04.496663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:04.499678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 377/758] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:04.505080 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999b932-3421-4f60-bfdd-1973636572c2 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:04.535482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0a6cb-c6e8-40a7-a25a-886490dce3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:04.535857 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0a6cb-c6e8-40a7-a25a-886490dce3c3 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:04.536140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0a6cb-c6e8-40a7-a25a-886490dce3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:04.536415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0a6cb-c6e8-40a7-a25a-886490dce3c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:04.549516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0a6cb-c6e8-40a7-a25a-886490dce3c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-45a0a6cb-c6e8-40a7-a25a-886490dce3c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:04.558784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:04.559764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:04.561388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:04.561388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:04.561388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:04.561388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:04.561388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:04.561388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:04.561388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:04.561388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:04.561388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:04.561388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:04.561388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:04.561388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:04.561388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 382/759] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:39:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:04.566175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0a6cb-c6e8-40a7-a25a-886490dce3c3 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:04.686596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0a6cb-c6e8-40a7-a25a-886490dce3c3 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:04.697044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999b932-3421-4f60-bfdd-1973636572c2 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:04.753110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978ac77-8fee-410a-970f-7a991928dc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:04.753803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978ac77-8fee-410a-970f-7a991928dc31 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:04.754062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978ac77-8fee-410a-970f-7a991928dc31 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:04.754343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978ac77-8fee-410a-970f-7a991928dc31 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:04.756177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978ac77-8fee-410a-970f-7a991928dc31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:04.756491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978ac77-8fee-410a-970f-7a991928dc31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:04.757886 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978ac77-8fee-410a-970f-7a991928dc31 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:04.768667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fec804-cf4c-465f-ae37-d2eca2b31882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:04.769330 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fec804-cf4c-465f-ae37-d2eca2b31882 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:04.769615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fec804-cf4c-465f-ae37-d2eca2b31882 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:04.769893 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fec804-cf4c-465f-ae37-d2eca2b31882 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:04.770616 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fec804-cf4c-465f-ae37-d2eca2b31882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:04.770947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fec804-cf4c-465f-ae37-d2eca2b31882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:04.772315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fec804-cf4c-465f-ae37-d2eca2b31882 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:04.776833 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 383/760] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:39:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:04.783695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978ac77-8fee-410a-970f-7a991928dc31 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:04.801430 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 378/761] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:04.806277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fec804-cf4c-465f-ae37-d2eca2b31882 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:04.916831 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fec804-cf4c-465f-ae37-d2eca2b31882 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:04.968685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e03c1b-4048-4e55-9b95-04fc4a51f67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:04.969488 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e03c1b-4048-4e55-9b95-04fc4a51f67f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:04.969771 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e03c1b-4048-4e55-9b95-04fc4a51f67f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:04.970053 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e03c1b-4048-4e55-9b95-04fc4a51f67f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:04.984590 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e03c1b-4048-4e55-9b95-04fc4a51f67f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:04.986062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e03c1b-4048-4e55-9b95-04fc4a51f67f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:04.988580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978ac77-8fee-410a-970f-7a991928dc31 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-43e03c1b-4048-4e55-9b95-04fc4a51f67f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:05.011609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:05.012558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:05.013983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:05.013983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:05.013983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:05.013983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:05.013983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:05.013983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:05.013983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:05.013983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:05.021208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 379/762] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:05.022089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e03c1b-4048-4e55-9b95-04fc4a51f67f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:05.054005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5873490-afcd-4779-a87e-2a8e64161e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:05.054731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5873490-afcd-4779-a87e-2a8e64161e87 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:05.055026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5873490-afcd-4779-a87e-2a8e64161e87 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:05.055307 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5873490-afcd-4779-a87e-2a8e64161e87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:05.074433 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5873490-afcd-4779-a87e-2a8e64161e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:05.076448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5873490-afcd-4779-a87e-2a8e64161e87 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-d5873490-afcd-4779-a87e-2a8e64161e87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:05.106081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:05.107107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:05.108601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:05.108601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:05.108601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:05.108601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:05.108601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 384/763] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:05.119699 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5873490-afcd-4779-a87e-2a8e64161e87 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:05.221471 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e03c1b-4048-4e55-9b95-04fc4a51f67f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:05.261609 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5873490-afcd-4779-a87e-2a8e64161e87 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:05.296098 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33bc62-b002-48bc-b7bd-fa0904c234b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:05.296869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33bc62-b002-48bc-b7bd-fa0904c234b6 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:05.297163 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33bc62-b002-48bc-b7bd-fa0904c234b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:05.297443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33bc62-b002-48bc-b7bd-fa0904c234b6 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:05.298185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33bc62-b002-48bc-b7bd-fa0904c234b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:05.298496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33bc62-b002-48bc-b7bd-fa0904c234b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:05.300077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33bc62-b002-48bc-b7bd-fa0904c234b6 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:05.300960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009733e6-193e-43b4-9b21-94854ec73f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:05.302215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009733e6-193e-43b4-9b21-94854ec73f3c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:05.302676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009733e6-193e-43b4-9b21-94854ec73f3c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:05.303172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009733e6-193e-43b4-9b21-94854ec73f3c None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:05.304232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009733e6-193e-43b4-9b21-94854ec73f3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:05.304754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009733e6-193e-43b4-9b21-94854ec73f3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:05.306487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009733e6-193e-43b4-9b21-94854ec73f3c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:05.314590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 385/764] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:05.321267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33bc62-b002-48bc-b7bd-fa0904c234b6 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:05.345301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 380/765] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:05.351454 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009733e6-193e-43b4-9b21-94854ec73f3c None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:05.422698 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33bc62-b002-48bc-b7bd-fa0904c234b6 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:05.464329 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153bcab-c4ec-402d-b0bb-e67255b1d0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:05.465121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153bcab-c4ec-402d-b0bb-e67255b1d0bb None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:05.465402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153bcab-c4ec-402d-b0bb-e67255b1d0bb None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:05.465674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153bcab-c4ec-402d-b0bb-e67255b1d0bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:05.477634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153bcab-c4ec-402d-b0bb-e67255b1d0bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:05.479106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153bcab-c4ec-402d-b0bb-e67255b1d0bb None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-8153bcab-c4ec-402d-b0bb-e67255b1d0bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:05.504001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:05.505071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:05.506000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:05.506000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:05.506000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:05.506000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:05.506000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:05.506000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:05.506000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:05.506000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:05.506000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 21:39:05.506000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:05.506000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 386/766] 10.222.0.35 () {56 vars in 1093 bytes} [Fri May 13 21:39:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:05.515559 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153bcab-c4ec-402d-b0bb-e67255b1d0bb None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:05.552543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009733e6-193e-43b4-9b21-94854ec73f3c None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:05.586793 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf7776-3853-46e6-81ca-9dde478196cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:05.587371 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf7776-3853-46e6-81ca-9dde478196cf None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:05.587649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf7776-3853-46e6-81ca-9dde478196cf None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:05.587919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf7776-3853-46e6-81ca-9dde478196cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:05.588744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf7776-3853-46e6-81ca-9dde478196cf None admin] RBAC: Adding query filter params (role.id=deb38f497be14e78ae21e8dceca3772f, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=a079ae1557d04831b73c03c96e2c7ad6) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:05.589052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf7776-3853-46e6-81ca-9dde478196cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:05.591877 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf7776-3853-46e6-81ca-9dde478196cf None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:05.598476 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 381/767] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:39:05 2022] GET /identity/v3/role_assignments?user.id=a079ae1557d04831b73c03c96e2c7ad6&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=deb38f497be14e78ae21e8dceca3772f => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:05.599951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b6b57-128d-4c6b-a8f7-bb41c8fe09d3 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:05.600239 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b6b57-128d-4c6b-a8f7-bb41c8fe09d3 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:05.600511 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b6b57-128d-4c6b-a8f7-bb41c8fe09d3 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:05.602101 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 382/768] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:05.608693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cde3c-f089-4fa9-8b91-35e89a077737 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:05.609049 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cde3c-f089-4fa9-8b91-35e89a077737 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:05.609239 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cde3c-f089-4fa9-8b91-35e89a077737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:05.628039 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-586cde3c-f089-4fa9-8b91-35e89a077737 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:05.629224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-586cde3c-f089-4fa9-8b91-35e89a077737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:05.637825 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153bcab-c4ec-402d-b0bb-e67255b1d0bb None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:05.680234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf28706-0ed7-4f89-81b9-75561c5011a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:05.680973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf28706-0ed7-4f89-81b9-75561c5011a5 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:05.681261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf28706-0ed7-4f89-81b9-75561c5011a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:05.681534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf28706-0ed7-4f89-81b9-75561c5011a5 None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:05.682397 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf28706-0ed7-4f89-81b9-75561c5011a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:05.682727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf28706-0ed7-4f89-81b9-75561c5011a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:05.684209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf28706-0ed7-4f89-81b9-75561c5011a5 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:05.704062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 387/769] 10.222.0.35 () {56 vars in 1133 bytes} [Fri May 13 21:39:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:05.712069 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf28706-0ed7-4f89-81b9-75561c5011a5 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:05.734858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 383/770] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:05.756700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cde3c-f089-4fa9-8b91-35e89a077737 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:05.861974 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cde3c-f089-4fa9-8b91-35e89a077737 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:05.896801 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbaa11-e17c-4390-8630-8d528fdad8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:05.897600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbaa11-e17c-4390-8630-8d528fdad8cb None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:05.897883 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbaa11-e17c-4390-8630-8d528fdad8cb None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:05.898152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbaa11-e17c-4390-8630-8d528fdad8cb None admin] PATH_INFO: `/v3/services/network` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:05.898835 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbaa11-e17c-4390-8630-8d528fdad8cb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:05.900209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbaa11-e17c-4390-8630-8d528fdad8cb None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:05.901107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf28706-0ed7-4f89-81b9-75561c5011a5 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-a4cbaa11-e17c-4390-8630-8d528fdad8cb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:05.912512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:05.913530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:05.914383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:05.914383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:05.914383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:05.914383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:05.914383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:05.914383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:05.914383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:05.914383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:05.914383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:05.914383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:05.914383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:05.914383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:05.914383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 21:39:05.914383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:05.918683 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 384/771] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:05 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:05.920119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbaa11-e17c-4390-8630-8d528fdad8cb None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:05.992319 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e8040-59e8-4e31-a13e-fd3f57c41224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:05.993491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e8040-59e8-4e31-a13e-fd3f57c41224 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:05.993983 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e8040-59e8-4e31-a13e-fd3f57c41224 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:05.994458 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e8040-59e8-4e31-a13e-fd3f57c41224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:06.014835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e8040-59e8-4e31-a13e-fd3f57c41224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:06.016639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e8040-59e8-4e31-a13e-fd3f57c41224 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:06.021756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbaa11-e17c-4390-8630-8d528fdad8cb None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-c96e8040-59e8-4e31-a13e-fd3f57c41224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:06.038884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:06.039795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:06.040759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:06.040759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:06.040759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:06.040759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:06.047034 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 388/772] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:06.049156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e8040-59e8-4e31-a13e-fd3f57c41224 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:06.067834 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613dcaf-59d9-4ce2-832d-f2d6f342256a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:06.068505 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613dcaf-59d9-4ce2-832d-f2d6f342256a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:06.068787 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613dcaf-59d9-4ce2-832d-f2d6f342256a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:06.069076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613dcaf-59d9-4ce2-832d-f2d6f342256a None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:06.070800 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613dcaf-59d9-4ce2-832d-f2d6f342256a None admin] RBAC: Adding query filter params (name=network) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:06.071564 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613dcaf-59d9-4ce2-832d-f2d6f342256a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:06.072971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613dcaf-59d9-4ce2-832d-f2d6f342256a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:06.081931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 385/773] 10.222.0.35 () {56 vars in 1112 bytes} [Fri May 13 21:39:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:06.088814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613dcaf-59d9-4ce2-832d-f2d6f342256a None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:06.223278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613dcaf-59d9-4ce2-832d-f2d6f342256a None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:06.235884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e8040-59e8-4e31-a13e-fd3f57c41224 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:06.291014 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c8a50-7476-4ed3-bd71-64c5a85cd075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:06.291673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c8a50-7476-4ed3-bd71-64c5a85cd075 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:06.291958 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c8a50-7476-4ed3-bd71-64c5a85cd075 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:06.292233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c8a50-7476-4ed3-bd71-64c5a85cd075 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:06.292949 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c8a50-7476-4ed3-bd71-64c5a85cd075 None admin] RBAC: Adding query filter params (type=network) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:06.293261 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c8a50-7476-4ed3-bd71-64c5a85cd075 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:06.294661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c8a50-7476-4ed3-bd71-64c5a85cd075 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:06.301030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728a228-9b62-4149-95a9-044f997cf81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:06.301698 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728a228-9b62-4149-95a9-044f997cf81b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:06.301984 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728a228-9b62-4149-95a9-044f997cf81b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:06.302261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728a228-9b62-4149-95a9-044f997cf81b None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:06.303008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728a228-9b62-4149-95a9-044f997cf81b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:06.303318 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728a228-9b62-4149-95a9-044f997cf81b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:06.304675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728a228-9b62-4149-95a9-044f997cf81b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:06.308157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 386/774] 10.222.0.35 () {56 vars in 1112 bytes} [Fri May 13 21:39:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:06.320994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c8a50-7476-4ed3-bd71-64c5a85cd075 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:06.334425 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 389/775] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:06.340212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728a228-9b62-4149-95a9-044f997cf81b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:06.447653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728a228-9b62-4149-95a9-044f997cf81b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:06.487024 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0962e9f-3edb-4abc-999f-fb7279db456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:06.488175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0962e9f-3edb-4abc-999f-fb7279db456d None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:06.488690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0962e9f-3edb-4abc-999f-fb7279db456d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:06.489167 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0962e9f-3edb-4abc-999f-fb7279db456d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:06.496743 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0962e9f-3edb-4abc-999f-fb7279db456d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:06.499030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0962e9f-3edb-4abc-999f-fb7279db456d None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-c0962e9f-3edb-4abc-999f-fb7279db456d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:06.514044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:06.515040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:06.516005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:06.516005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:06.516005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:06.516005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:06.516005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:06.516005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:06.516005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:06.516005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:06.523195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 390/776] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:06.529006 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0962e9f-3edb-4abc-999f-fb7279db456d None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:06.550941 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c8a50-7476-4ed3-bd71-64c5a85cd075 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:06.637332 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0962e9f-3edb-4abc-999f-fb7279db456d None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:06.644171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8c503-d422-493c-ac7c-b05d093942cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:06.644840 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8c503-d422-493c-ac7c-b05d093942cb None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:06.645123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8c503-d422-493c-ac7c-b05d093942cb None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:06.645404 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8c503-d422-493c-ac7c-b05d093942cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:06.646199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8c503-d422-493c-ac7c-b05d093942cb None admin] RBAC: Adding query filter params (interface=public, service_id=6132b1f3234b4a1393a48fa857d61b95, region_id=RegionOne) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:06.646513 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8c503-d422-493c-ac7c-b05d093942cb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:06.647917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8c503-d422-493c-ac7c-b05d093942cb None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:06.665062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 387/777] 10.222.0.35 () {56 vars in 1251 bytes} [Fri May 13 21:39:06 2022] GET /identity/v3/endpoints?service_id=6132b1f3234b4a1393a48fa857d61b95&interface=public®ion_id=RegionOne => generated 194 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:06.670962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8c503-d422-493c-ac7c-b05d093942cb None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:06.676136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e0ed5-5776-47c0-8ed5-9095cd40324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:06.677181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e0ed5-5776-47c0-8ed5-9095cd40324c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:06.677673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e0ed5-5776-47c0-8ed5-9095cd40324c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:06.678189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e0ed5-5776-47c0-8ed5-9095cd40324c None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:06.679271 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e0ed5-5776-47c0-8ed5-9095cd40324c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:06.679871 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e0ed5-5776-47c0-8ed5-9095cd40324c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:06.681634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e0ed5-5776-47c0-8ed5-9095cd40324c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:06.696559 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 391/778] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:06.703052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e0ed5-5776-47c0-8ed5-9095cd40324c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:06.817967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e0ed5-5776-47c0-8ed5-9095cd40324c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:06.855573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f9734-fe11-4cb9-acb6-3e8d2317fc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:06.857137 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f9734-fe11-4cb9-acb6-3e8d2317fc0f None admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:06.858153 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f9734-fe11-4cb9-acb6-3e8d2317fc0f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:06.858645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f9734-fe11-4cb9-acb6-3e8d2317fc0f None admin] PATH_INFO: `/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/316ef43c6c304bd3a27e085bf4da8a5e/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:06.861667 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f9734-fe11-4cb9-acb6-3e8d2317fc0f None admin] RBAC: Authorizing `identity:create_grant(project_id=ac834e84b2c84261ade311a6dfbbeb20, user_id=316ef43c6c304bd3a27e085bf4da8a5e, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:06.863371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f9734-fe11-4cb9-acb6-3e8d2317fc0f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:06.880917 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 392/779] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:39:06 2022] PUT /identity/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/316ef43c6c304bd3a27e085bf4da8a5e/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:39:06.887500 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ba209-adb6-4975-82fa-667c271303a1 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:06.888088 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ba209-adb6-4975-82fa-667c271303a1 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:06.888611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ba209-adb6-4975-82fa-667c271303a1 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:06.890541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 393/780] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:06.896641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a888b4-d757-4856-9d23-b0bea3e4f1b3 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:06.897266 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a888b4-d757-4856-9d23-b0bea3e4f1b3 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:06.897768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a888b4-d757-4856-9d23-b0bea3e4f1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:06.900052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8c503-d422-493c-ac7c-b05d093942cb None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:06.927401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-e0a888b4-d757-4856-9d23-b0bea3e4f1b3 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:06.929357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-e0a888b4-d757-4856-9d23-b0bea3e4f1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:06.945617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee72b09-5f6e-499b-86b8-7863d4147382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:06.946293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee72b09-5f6e-499b-86b8-7863d4147382 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:06.946573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee72b09-5f6e-499b-86b8-7863d4147382 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:06.946877 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee72b09-5f6e-499b-86b8-7863d4147382 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:06.947567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee72b09-5f6e-499b-86b8-7863d4147382 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:06.948986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee72b09-5f6e-499b-86b8-7863d4147382 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:06.957369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 388/781] 10.222.0.35 () {56 vars in 1087 bytes} [Fri May 13 21:39:06 2022] GET /identity/v3/services => generated 772 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:07.048711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 394/782] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:07.084620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee72b09-5f6e-499b-86b8-7863d4147382 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:07.212925 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee72b09-5f6e-499b-86b8-7863d4147382 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:07.217614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52de36-978f-49b1-968c-e4d208fe44bb None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:07.217906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52de36-978f-49b1-968c-e4d208fe44bb None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:07.218188 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52de36-978f-49b1-968c-e4d208fe44bb None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:07.220896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 395/783] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:07.225144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c8b4-ae0c-47d2-a146-4552f32781d8 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:07.225433 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c8b4-ae0c-47d2-a146-4552f32781d8 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:07.225712 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c8b4-ae0c-47d2-a146-4552f32781d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:07.245359 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-9282c8b4-ae0c-47d2-a146-4552f32781d8 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:07.246551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-9282c8b4-ae0c-47d2-a146-4552f32781d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:07.264215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7de46-2013-476e-8a0f-fa4b991e1a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:07.265015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7de46-2013-476e-8a0f-fa4b991e1a54 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:07.265300 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7de46-2013-476e-8a0f-fa4b991e1a54 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:07.266171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7de46-2013-476e-8a0f-fa4b991e1a54 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:07.266292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7de46-2013-476e-8a0f-fa4b991e1a54 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:07.267686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7de46-2013-476e-8a0f-fa4b991e1a54 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-09c7de46-2013-476e-8a0f-fa4b991e1a54 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:07.280369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:07.281362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:07.282228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 389/784] 10.222.0.35 () {56 vars in 1129 bytes} [Fri May 13 21:39:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:07.290993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7de46-2013-476e-8a0f-fa4b991e1a54 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:07.331090 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 396/785] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:07.354032 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282c8b4-ae0c-47d2-a146-4552f32781d8 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:07.424481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7de46-2013-476e-8a0f-fa4b991e1a54 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:07.460163 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282c8b4-ae0c-47d2-a146-4552f32781d8 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:07.494696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ec374-dc35-4115-ba3d-aebb9ef76e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:07.496104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97e849-968b-43e7-b8c6-063bf1d5d583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:07.496888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97e849-968b-43e7-b8c6-063bf1d5d583 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:07.497174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97e849-968b-43e7-b8c6-063bf1d5d583 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:07.497453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97e849-968b-43e7-b8c6-063bf1d5d583 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:07.498470 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ec374-dc35-4115-ba3d-aebb9ef76e28 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:07.498766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ec374-dc35-4115-ba3d-aebb9ef76e28 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:07.499049 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ec374-dc35-4115-ba3d-aebb9ef76e28 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:07.499765 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ec374-dc35-4115-ba3d-aebb9ef76e28 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:07.500076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ec374-dc35-4115-ba3d-aebb9ef76e28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:07.501462 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ec374-dc35-4115-ba3d-aebb9ef76e28 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:07.507644 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 390/786] 10.222.0.35 () {56 vars in 1124 bytes} [Fri May 13 21:39:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:07.512663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ec374-dc35-4115-ba3d-aebb9ef76e28 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:07.527606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97e849-968b-43e7-b8c6-063bf1d5d583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-0f97e849-968b-43e7-b8c6-063bf1d5d583 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:07.535866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:07.536803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:07.537658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:07.537658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:07.537658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:07.537658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:07.537658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:07.537658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:07.537658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:07.537658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:07.537658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:07.537658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:07.537658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:07.537658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:07.537658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 397/787] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:39:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:07.542950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97e849-968b-43e7-b8c6-063bf1d5d583 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:07.693028 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ec374-dc35-4115-ba3d-aebb9ef76e28 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:07.716008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97e849-968b-43e7-b8c6-063bf1d5d583 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:07.753388 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad7b7f-cfd7-493a-ac0b-269b62c678a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:07.753984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad7b7f-cfd7-493a-ac0b-269b62c678a8 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:07.754381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad7b7f-cfd7-493a-ac0b-269b62c678a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:07.754665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad7b7f-cfd7-493a-ac0b-269b62c678a8 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:07.755407 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad7b7f-cfd7-493a-ac0b-269b62c678a8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:07.755718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad7b7f-cfd7-493a-ac0b-269b62c678a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:07.757110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad7b7f-cfd7-493a-ac0b-269b62c678a8 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:07.765252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 391/788] 10.222.0.35 () {56 vars in 1124 bytes} [Fri May 13 21:39:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:07.772399 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad7b7f-cfd7-493a-ac0b-269b62c678a8 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:07.774170 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09a775-ef75-47bc-bd21-521052900c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:07.774865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09a775-ef75-47bc-bd21-521052900c35 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:07.775147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09a775-ef75-47bc-bd21-521052900c35 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:07.775420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09a775-ef75-47bc-bd21-521052900c35 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:07.776165 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09a775-ef75-47bc-bd21-521052900c35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:07.776469 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09a775-ef75-47bc-bd21-521052900c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:07.777835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09a775-ef75-47bc-bd21-521052900c35 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:07.806936 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 398/789] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:39:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:07.807863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09a775-ef75-47bc-bd21-521052900c35 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:07.923872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad7b7f-cfd7-493a-ac0b-269b62c678a8 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:07.982195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53cd68-73ab-4cd1-a37b-1b64d873c406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:07.982883 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53cd68-73ab-4cd1-a37b-1b64d873c406 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:07.983216 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53cd68-73ab-4cd1-a37b-1b64d873c406 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:07.983497 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53cd68-73ab-4cd1-a37b-1b64d873c406 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:07.984240 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53cd68-73ab-4cd1-a37b-1b64d873c406 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:07.985612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53cd68-73ab-4cd1-a37b-1b64d873c406 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:07.998230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09a775-ef75-47bc-bd21-521052900c35 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:08.035906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb2f94-c61b-4ce2-96ab-62d8902e7faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:08.037773 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb2f94-c61b-4ce2-96ab-62d8902e7faa None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:08.038064 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb2f94-c61b-4ce2-96ab-62d8902e7faa None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:08.038346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb2f94-c61b-4ce2-96ab-62d8902e7faa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:08.040742 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 392/790] 10.222.0.35 () {60 vars in 1144 bytes} [Fri May 13 21:39:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:39:08.052698 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb2f94-c61b-4ce2-96ab-62d8902e7faa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:08.054225 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb2f94-c61b-4ce2-96ab-62d8902e7faa None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-5beb2f94-c61b-4ce2-96ab-62d8902e7faa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:08.072704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:08.073679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:08.074575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:08.074575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:08.074575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:08.074575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:08.074575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 399/791] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:08.080912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53cd68-73ab-4cd1-a37b-1b64d873c406 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:08.202181 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53cd68-73ab-4cd1-a37b-1b64d873c406 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:08.238761 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e18b1-f842-469b-bf5a-3b08c7d6d1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:08.239440 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e18b1-f842-469b-bf5a-3b08c7d6d1dd None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:08.239719 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e18b1-f842-469b-bf5a-3b08c7d6d1dd None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:08.239999 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e18b1-f842-469b-bf5a-3b08c7d6d1dd None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:08.240735 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e18b1-f842-469b-bf5a-3b08c7d6d1dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:08.241043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e18b1-f842-469b-bf5a-3b08c7d6d1dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:08.242433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e18b1-f842-469b-bf5a-3b08c7d6d1dd None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:08.257231 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 393/792] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:08.262853 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb2f94-c61b-4ce2-96ab-62d8902e7faa None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:08.303579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3eb8b2-6797-4d23-8d60-85cb6cb43b42 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:08.304219 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3eb8b2-6797-4d23-8d60-85cb6cb43b42 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:08.304713 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3eb8b2-6797-4d23-8d60-85cb6cb43b42 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:08.306650 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 394/793] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:39:08.313120 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4928c-a666-4544-ae35-da4043cbc652 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:08.313696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4928c-a666-4544-ae35-da4043cbc652 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:08.314208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4928c-a666-4544-ae35-da4043cbc652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:08.331643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a0e4928c-a666-4544-ae35-da4043cbc652 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:08.333114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a0e4928c-a666-4544-ae35-da4043cbc652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:08.368489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb2f94-c61b-4ce2-96ab-62d8902e7faa None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:08.401664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 395/794] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:08.406733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f54f1c-9cc4-44de-befd-d343bb50b87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:08.406733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f54f1c-9cc4-44de-befd-d343bb50b87c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:08.406733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f54f1c-9cc4-44de-befd-d343bb50b87c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:08.406733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f54f1c-9cc4-44de-befd-d343bb50b87c None admin] PATH_INFO: `/v3/users/swift` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:08.412326 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f54f1c-9cc4-44de-befd-d343bb50b87c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:08.413773 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f54f1c-9cc4-44de-befd-d343bb50b87c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-c0f54f1c-9cc4-44de-befd-d343bb50b87c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:08.428574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:08.429535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:08.430483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:08.430483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:08.430483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:08.430483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:08.430483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:08.430483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:08.430483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:08.430483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:08.430483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 21:39:08.430483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:08.430483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 400/795] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:39:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:08.432304 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4928c-a666-4544-ae35-da4043cbc652 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:08.435516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f54f1c-9cc4-44de-befd-d343bb50b87c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:08.532552 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4928c-a666-4544-ae35-da4043cbc652 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:08.539709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f54f1c-9cc4-44de-befd-d343bb50b87c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:08.568586 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c444f6-1e55-4432-a792-060435755066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:08.569738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c444f6-1e55-4432-a792-060435755066 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:08.570230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c444f6-1e55-4432-a792-060435755066 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:08.570754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c444f6-1e55-4432-a792-060435755066 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:08.573928 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126724a-af26-4bf2-950f-89518250bf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:08.574895 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126724a-af26-4bf2-950f-89518250bf3f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:08.575356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126724a-af26-4bf2-950f-89518250bf3f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:08.575819 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126724a-af26-4bf2-950f-89518250bf3f None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:08.576890 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126724a-af26-4bf2-950f-89518250bf3f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:08.577356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126724a-af26-4bf2-950f-89518250bf3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:08.579033 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126724a-af26-4bf2-950f-89518250bf3f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:08.583893 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c444f6-1e55-4432-a792-060435755066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-70c444f6-1e55-4432-a792-060435755066 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:08.592405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:08.593395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:08.594245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:08.594245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:08.594245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:08.594245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:08.594245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:08.594245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:08.594245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:08.594245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:08.594245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:08.594245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:08.594245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:39:08.594245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:08.594245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 396/796] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:39:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:08.599735 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c444f6-1e55-4432-a792-060435755066 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:08.602596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 401/797] 10.222.0.35 () {56 vars in 1135 bytes} [Fri May 13 21:39:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:08.608620 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126724a-af26-4bf2-950f-89518250bf3f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:08.717976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c444f6-1e55-4432-a792-060435755066 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:08.760590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126724a-af26-4bf2-950f-89518250bf3f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:08.761275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370d80d-bd76-4851-8a02-08d652312f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:08.762214 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370d80d-bd76-4851-8a02-08d652312f81 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:08.762686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370d80d-bd76-4851-8a02-08d652312f81 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:08.763702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370d80d-bd76-4851-8a02-08d652312f81 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:08.766242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370d80d-bd76-4851-8a02-08d652312f81 None admin] RBAC: Adding query filter params (name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:08.766756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370d80d-bd76-4851-8a02-08d652312f81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:08.769798 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370d80d-bd76-4851-8a02-08d652312f81 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:08.788224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 397/798] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:39:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:08.794412 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370d80d-bd76-4851-8a02-08d652312f81 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:08.838958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b6e29-2bbf-41f3-b2d6-c745971310a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:08.840066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b6e29-2bbf-41f3-b2d6-c745971310a6 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:08.842039 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b6e29-2bbf-41f3-b2d6-c745971310a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:08.843295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b6e29-2bbf-41f3-b2d6-c745971310a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:08.856184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b6e29-2bbf-41f3-b2d6-c745971310a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:08.858498 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b6e29-2bbf-41f3-b2d6-c745971310a6 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-e95b6e29-2bbf-41f3-b2d6-c745971310a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:08.894519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:08.895494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:08.896446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:08.896446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:08.896446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:08.896446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:08.896446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 402/799] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:08.903367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b6e29-2bbf-41f3-b2d6-c745971310a6 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:08.956908 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370d80d-bd76-4851-8a02-08d652312f81 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:09.000984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618ae70-7c07-4b4f-a55e-88c6b7bdeada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:09.002433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618ae70-7c07-4b4f-a55e-88c6b7bdeada None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:09.002959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618ae70-7c07-4b4f-a55e-88c6b7bdeada None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:09.003433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618ae70-7c07-4b4f-a55e-88c6b7bdeada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:09.027201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6618ae70-7c07-4b4f-a55e-88c6b7bdeada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:09.030493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6618ae70-7c07-4b4f-a55e-88c6b7bdeada None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-6618ae70-7c07-4b4f-a55e-88c6b7bdeada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:09.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:09.065870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:09.066774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:09.066774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:09.066774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:09.066774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:09.069184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 398/800] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:09.077490 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618ae70-7c07-4b4f-a55e-88c6b7bdeada None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:09.103540 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b6e29-2bbf-41f3-b2d6-c745971310a6 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:09.139298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cb340-568b-42e0-aa65-1b8ee09b0a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:09.139970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cb340-568b-42e0-aa65-1b8ee09b0a5b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:09.140302 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cb340-568b-42e0-aa65-1b8ee09b0a5b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:09.140525 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cb340-568b-42e0-aa65-1b8ee09b0a5b None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:09.141253 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4cb340-568b-42e0-aa65-1b8ee09b0a5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:09.141562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4cb340-568b-42e0-aa65-1b8ee09b0a5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:09.142991 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4cb340-568b-42e0-aa65-1b8ee09b0a5b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:09.160548 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 403/801] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:09.176196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cb340-568b-42e0-aa65-1b8ee09b0a5b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:09.194573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618ae70-7c07-4b4f-a55e-88c6b7bdeada None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:09.235548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3225c-e852-411a-a8ce-59cd8949e20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:09.236609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3225c-e852-411a-a8ce-59cd8949e20e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:09.237119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3225c-e852-411a-a8ce-59cd8949e20e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:09.237596 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3225c-e852-411a-a8ce-59cd8949e20e None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:09.238520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3225c-e852-411a-a8ce-59cd8949e20e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:09.239155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3225c-e852-411a-a8ce-59cd8949e20e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:09.240744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3225c-e852-411a-a8ce-59cd8949e20e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:09.264153 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 399/802] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:09.275698 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3225c-e852-411a-a8ce-59cd8949e20e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:09.379602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cb340-568b-42e0-aa65-1b8ee09b0a5b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:09.433432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3225c-e852-411a-a8ce-59cd8949e20e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:09.442733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63eec8-ba23-4a1f-b13f-b975fbb2fd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:09.443579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63eec8-ba23-4a1f-b13f-b975fbb2fd80 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:09.443861 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63eec8-ba23-4a1f-b13f-b975fbb2fd80 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:09.444175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63eec8-ba23-4a1f-b13f-b975fbb2fd80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:09.454334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63eec8-ba23-4a1f-b13f-b975fbb2fd80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:09.455840 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63eec8-ba23-4a1f-b13f-b975fbb2fd80 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-8c63eec8-ba23-4a1f-b13f-b975fbb2fd80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:09.472243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:09.473261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:09.474174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:09.474174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:09.474174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:09.474174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:09.474174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:09.474174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:09.474174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:09.474174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:09.474603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 404/803] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:09.483782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63eec8-ba23-4a1f-b13f-b975fbb2fd80 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:09.492500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7c4ec-ee9a-47ce-807f-e33e8278ef4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:09.493325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7c4ec-ee9a-47ce-807f-e33e8278ef4c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:09.493604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7c4ec-ee9a-47ce-807f-e33e8278ef4c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:09.493876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7c4ec-ee9a-47ce-807f-e33e8278ef4c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:09.502649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7c4ec-ee9a-47ce-807f-e33e8278ef4c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:09.504126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7c4ec-ee9a-47ce-807f-e33e8278ef4c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-d9e7c4ec-ee9a-47ce-807f-e33e8278ef4c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:09.519432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:09.520401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:09.521324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:09.521324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:09.521324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:09.521324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:09.521324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:09.521324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:09.521324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:09.521324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:09.521324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 21:39:09.521324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:09.521324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 400/804] 10.222.0.35 () {56 vars in 1117 bytes} [Fri May 13 21:39:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:09.529938 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7c4ec-ee9a-47ce-807f-e33e8278ef4c None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:09.646812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7c4ec-ee9a-47ce-807f-e33e8278ef4c None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:09.687663 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63eec8-ba23-4a1f-b13f-b975fbb2fd80 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:09.705620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c30a8-1ec6-4b37-99c4-3d3c3a862e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:09.706306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c30a8-1ec6-4b37-99c4-3d3c3a862e5e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:09.706590 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c30a8-1ec6-4b37-99c4-3d3c3a862e5e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:09.706976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c30a8-1ec6-4b37-99c4-3d3c3a862e5e None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:09.707837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c30a8-1ec6-4b37-99c4-3d3c3a862e5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:09.708153 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c30a8-1ec6-4b37-99c4-3d3c3a862e5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:09.709660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c30a8-1ec6-4b37-99c4-3d3c3a862e5e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:09.725594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaac843-885e-4543-a3bb-8a34e771ffd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:09.726340 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaac843-885e-4543-a3bb-8a34e771ffd1 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:09.726641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaac843-885e-4543-a3bb-8a34e771ffd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:09.726921 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaac843-885e-4543-a3bb-8a34e771ffd1 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:09.727768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaac843-885e-4543-a3bb-8a34e771ffd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:09.728077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaac843-885e-4543-a3bb-8a34e771ffd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:09.729532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaac843-885e-4543-a3bb-8a34e771ffd1 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:09.729727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 401/805] 10.222.0.35 () {56 vars in 1149 bytes} [Fri May 13 21:39:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:09.731225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a00003-f567-4230-b232-5f846897d147 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:09.732357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a00003-f567-4230-b232-5f846897d147 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:09.732782 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a00003-f567-4230-b232-5f846897d147 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:09.738289 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 402/806] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:09.740027 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4346b-4644-4159-87ed-aa231e1819a3 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:09.740467 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4346b-4644-4159-87ed-aa231e1819a3 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:09.740914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4346b-4644-4159-87ed-aa231e1819a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:09.748074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 405/807] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:09.748959 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaac843-885e-4543-a3bb-8a34e771ffd1 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:09.759236 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-deb4346b-4644-4159-87ed-aa231e1819a3 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:09.760739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-deb4346b-4644-4159-87ed-aa231e1819a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:09.838973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 403/808] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:09.840250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4346b-4644-4159-87ed-aa231e1819a3 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:09.849843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaac843-885e-4543-a3bb-8a34e771ffd1 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:09.883997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6faf78-4493-4ecd-b639-425487e8567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:09.884779 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6faf78-4493-4ecd-b639-425487e8567b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:09.885059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6faf78-4493-4ecd-b639-425487e8567b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:09.885329 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6faf78-4493-4ecd-b639-425487e8567b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:09.896353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6faf78-4493-4ecd-b639-425487e8567b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:09.897851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6faf78-4493-4ecd-b639-425487e8567b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-ff6faf78-4493-4ecd-b639-425487e8567b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:09.915212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:09.916115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:09.917076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:09.917076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:09.917076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:09.917076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:09.917076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 406/809] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:09.917346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6faf78-4493-4ecd-b639-425487e8567b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:09.968983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4346b-4644-4159-87ed-aa231e1819a3 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:10.001099 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199619a-e9d3-468f-bf47-90876273b68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:10.001966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199619a-e9d3-468f-bf47-90876273b68d None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:10.002504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199619a-e9d3-468f-bf47-90876273b68d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:10.002812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199619a-e9d3-468f-bf47-90876273b68d None admin] PATH_INFO: `/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/4f9e07ed0a084c82b946aaf585de16fd/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:10.004755 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199619a-e9d3-468f-bf47-90876273b68d None admin] RBAC: Authorizing `identity:create_grant(project_id=ac834e84b2c84261ade311a6dfbbeb20, user_id=4f9e07ed0a084c82b946aaf585de16fd, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:10.006174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199619a-e9d3-468f-bf47-90876273b68d None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:10.018932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 404/810] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:39:09 2022] PUT /identity/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/4f9e07ed0a084c82b946aaf585de16fd/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:39:10.019067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6faf78-4493-4ecd-b639-425487e8567b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:10.020706 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28645026-d45e-4b8b-a684-6eb2b47ae76b None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:10.021209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28645026-d45e-4b8b-a684-6eb2b47ae76b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:10.021711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28645026-d45e-4b8b-a684-6eb2b47ae76b None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:10.027384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 405/811] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:10.028098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28645026-d45e-4b8b-a684-6eb2b47ae76b None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:10.073978 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13dc9e-aa2d-486e-9b91-bb1146850f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:10.075950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13dc9e-aa2d-486e-9b91-bb1146850f93 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:10.076451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13dc9e-aa2d-486e-9b91-bb1146850f93 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:10.076927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13dc9e-aa2d-486e-9b91-bb1146850f93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:10.092743 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13dc9e-aa2d-486e-9b91-bb1146850f93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-cc13dc9e-aa2d-486e-9b91-bb1146850f93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:10.102989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:10.105524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:10.106371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:10.106371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:10.106371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:10.106371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:10.106371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:10.106371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:10.106371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:10.106371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:10.106371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:10.106371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:10.106371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:10.106371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:10.106371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 407/812] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:39:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:10.106371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a3c10-940e-420b-96c9-d281e8d23f13 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:10.108427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a3c10-940e-420b-96c9-d281e8d23f13 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:10.108427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a3c10-940e-420b-96c9-d281e8d23f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:10.128538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-c57a3c10-940e-420b-96c9-d281e8d23f13 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:10.129848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c57a3c10-940e-420b-96c9-d281e8d23f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:10.149959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28645026-d45e-4b8b-a684-6eb2b47ae76b None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:10.200525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038de86e-f4aa-49b1-8a7d-1d46f885e574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:10.202242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038de86e-f4aa-49b1-8a7d-1d46f885e574 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:10.202767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038de86e-f4aa-49b1-8a7d-1d46f885e574 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:10.203119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038de86e-f4aa-49b1-8a7d-1d46f885e574 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:10.203876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038de86e-f4aa-49b1-8a7d-1d46f885e574 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:10.204792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038de86e-f4aa-49b1-8a7d-1d46f885e574 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:10.206238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038de86e-f4aa-49b1-8a7d-1d46f885e574 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:10.215351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 408/813] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:10.217749 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a3c10-940e-420b-96c9-d281e8d23f13 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:10.223272 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 406/814] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:10.230951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038de86e-f4aa-49b1-8a7d-1d46f885e574 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:10.346691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038de86e-f4aa-49b1-8a7d-1d46f885e574 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:10.348191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a3c10-940e-420b-96c9-d281e8d23f13 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:10.381438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe7afe-5081-4362-8262-6a708beb838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:10.382482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe7afe-5081-4362-8262-6a708beb838a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:10.382768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7839d-95d6-4687-86d7-3916f485df01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:10.383138 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe7afe-5081-4362-8262-6a708beb838a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:10.383562 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe7afe-5081-4362-8262-6a708beb838a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:10.383839 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7839d-95d6-4687-86d7-3916f485df01 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:10.384217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7839d-95d6-4687-86d7-3916f485df01 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:10.384589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7839d-95d6-4687-86d7-3916f485df01 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:10.385433 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7839d-95d6-4687-86d7-3916f485df01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:10.385834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7839d-95d6-4687-86d7-3916f485df01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:10.387332 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7839d-95d6-4687-86d7-3916f485df01 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:10.392340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe7afe-5081-4362-8262-6a708beb838a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:10.393951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe7afe-5081-4362-8262-6a708beb838a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:10.396910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 409/815] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:39:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:10.397874 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7839d-95d6-4687-86d7-3916f485df01 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-fabe7afe-5081-4362-8262-6a708beb838a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:10.409948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:10.411063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:10.412179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:10.412179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:10.412179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:10.412179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:10.412179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:10.412179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:10.412179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:10.412179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:10.414306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 407/816] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:10.419744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe7afe-5081-4362-8262-6a708beb838a None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:10.501587 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7839d-95d6-4687-86d7-3916f485df01 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:10.520409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe7afe-5081-4362-8262-6a708beb838a None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:10.541299 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467f286-2b64-44b7-a806-5dc37f1529e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:10.542872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467f286-2b64-44b7-a806-5dc37f1529e5 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:10.543495 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467f286-2b64-44b7-a806-5dc37f1529e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:10.544110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467f286-2b64-44b7-a806-5dc37f1529e5 None admin] PATH_INFO: `/v3/services/network` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:10.545520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467f286-2b64-44b7-a806-5dc37f1529e5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:10.547214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467f286-2b64-44b7-a806-5dc37f1529e5 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-5467f286-2b64-44b7-a806-5dc37f1529e5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:10.559585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:10.561655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:10.562492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 410/817] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:10 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:10.564290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467f286-2b64-44b7-a806-5dc37f1529e5 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:10.571005 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d9e82-7507-41f4-941b-18bad0ad1888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:10.572097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d9e82-7507-41f4-941b-18bad0ad1888 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:10.572579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d9e82-7507-41f4-941b-18bad0ad1888 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:10.573037 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d9e82-7507-41f4-941b-18bad0ad1888 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:10.584442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d9e82-7507-41f4-941b-18bad0ad1888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:10.586146 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d9e82-7507-41f4-941b-18bad0ad1888 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-0d6d9e82-7507-41f4-941b-18bad0ad1888 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:10.604553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:10.605573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:10.606549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:10.606549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:10.606549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:10.606549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:10.608767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 408/818] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:10.609830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d9e82-7507-41f4-941b-18bad0ad1888 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:10.685453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467f286-2b64-44b7-a806-5dc37f1529e5 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:10.714665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d9e82-7507-41f4-941b-18bad0ad1888 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:10.733758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852cb95-c9f6-4a48-bab7-7ae8fd11388d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:10.734422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852cb95-c9f6-4a48-bab7-7ae8fd11388d None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:10.734727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852cb95-c9f6-4a48-bab7-7ae8fd11388d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:10.735011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852cb95-c9f6-4a48-bab7-7ae8fd11388d None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:10.735799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852cb95-c9f6-4a48-bab7-7ae8fd11388d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:10.736195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852cb95-c9f6-4a48-bab7-7ae8fd11388d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:10.738411 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852cb95-c9f6-4a48-bab7-7ae8fd11388d None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:10.750343 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8efa23-3c24-4726-a7ec-b5e8c6947a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:10.751282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8efa23-3c24-4726-a7ec-b5e8c6947a6e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:10.751760 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8efa23-3c24-4726-a7ec-b5e8c6947a6e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:10.752468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8efa23-3c24-4726-a7ec-b5e8c6947a6e None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:10.753336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8efa23-3c24-4726-a7ec-b5e8c6947a6e None admin] RBAC: Adding query filter params (name=network) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:10.753811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8efa23-3c24-4726-a7ec-b5e8c6947a6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:10.755438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8efa23-3c24-4726-a7ec-b5e8c6947a6e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:10.758669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 411/819] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:10.759867 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852cb95-c9f6-4a48-bab7-7ae8fd11388d None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:10.762904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 409/820] 10.222.0.35 () {56 vars in 1112 bytes} [Fri May 13 21:39:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:10.764067 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8efa23-3c24-4726-a7ec-b5e8c6947a6e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:10.867140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852cb95-c9f6-4a48-bab7-7ae8fd11388d None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:10.891086 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8efa23-3c24-4726-a7ec-b5e8c6947a6e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:10.906026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd170dd-0089-4d96-8433-39e83aa4eefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:10.906851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd170dd-0089-4d96-8433-39e83aa4eefb None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:10.907136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd170dd-0089-4d96-8433-39e83aa4eefb None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:10.907413 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd170dd-0089-4d96-8433-39e83aa4eefb None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:10.908313 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd170dd-0089-4d96-8433-39e83aa4eefb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:10.908996 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd170dd-0089-4d96-8433-39e83aa4eefb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:10.910409 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd170dd-0089-4d96-8433-39e83aa4eefb None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:10.927259 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 412/821] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:10.927259 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd170dd-0089-4d96-8433-39e83aa4eefb None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:10.933316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03999c54-5c35-456f-89e3-7c7c78ec7bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:10.934199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03999c54-5c35-456f-89e3-7c7c78ec7bbb None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:10.934476 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03999c54-5c35-456f-89e3-7c7c78ec7bbb None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:10.934775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03999c54-5c35-456f-89e3-7c7c78ec7bbb None admin] PATH_INFO: `/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/a079ae1557d04831b73c03c96e2c7ad6/roles/deb38f497be14e78ae21e8dceca3772f` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:10.949841 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03999c54-5c35-456f-89e3-7c7c78ec7bbb None admin] RBAC: Authorizing `identity:create_grant(project_id=ac834e84b2c84261ade311a6dfbbeb20, user_id=a079ae1557d04831b73c03c96e2c7ad6, role_id=deb38f497be14e78ae21e8dceca3772f)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:10.951774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03999c54-5c35-456f-89e3-7c7c78ec7bbb None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:10.965214 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 410/822] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:39:10 2022] PUT /identity/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/a079ae1557d04831b73c03c96e2c7ad6/roles/deb38f497be14e78ae21e8dceca3772f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:39:10.975007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3375c-2ba3-4ed7-8326-40a93a4a5895 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:10.975007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3375c-2ba3-4ed7-8326-40a93a4a5895 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:10.975007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3375c-2ba3-4ed7-8326-40a93a4a5895 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:10.975007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 411/823] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:10.975007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3375c-2ba3-4ed7-8326-40a93a4a5895 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:11.051440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd170dd-0089-4d96-8433-39e83aa4eefb None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:11.094844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c8d0b-d0fa-4ea1-bf2b-097dcd69cfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:11.095670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c8d0b-d0fa-4ea1-bf2b-097dcd69cfd3 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:11.096056 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c8d0b-d0fa-4ea1-bf2b-097dcd69cfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:11.096535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c8d0b-d0fa-4ea1-bf2b-097dcd69cfd3 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:11.097417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c8d0b-d0fa-4ea1-bf2b-097dcd69cfd3 None admin] RBAC: Adding query filter params (type=network) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:11.097824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c8d0b-d0fa-4ea1-bf2b-097dcd69cfd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:11.099421 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c8d0b-d0fa-4ea1-bf2b-097dcd69cfd3 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:11.101465 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3375c-2ba3-4ed7-8326-40a93a4a5895 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:11.105493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 413/824] 10.222.0.35 () {56 vars in 1112 bytes} [Fri May 13 21:39:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:11.107211 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c886b-5351-4abb-89ec-4839a17188a9 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:11.107602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c886b-5351-4abb-89ec-4839a17188a9 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:11.107986 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c886b-5351-4abb-89ec-4839a17188a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:11.131574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-4c7c886b-5351-4abb-89ec-4839a17188a9 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:11.133592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4c7c886b-5351-4abb-89ec-4839a17188a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:11.141413 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ab51e-ff4f-4d8a-b75c-35ca7a6515b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:11.142296 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ab51e-ff4f-4d8a-b75c-35ca7a6515b8 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:11.142585 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ab51e-ff4f-4d8a-b75c-35ca7a6515b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:11.142910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ab51e-ff4f-4d8a-b75c-35ca7a6515b8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:11.153193 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ab51e-ff4f-4d8a-b75c-35ca7a6515b8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:11.158290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ab51e-ff4f-4d8a-b75c-35ca7a6515b8 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-c01ab51e-ff4f-4d8a-b75c-35ca7a6515b8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:11.180629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:11.181829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:11.183152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:11.183152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:11.183152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:11.183152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:11.183152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:11.183152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:11.183152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:11.183152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:11.183152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 21:39:11.183152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:11.183152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 412/825] 10.222.0.35 () {56 vars in 1093 bytes} [Fri May 13 21:39:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:11.183152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ab51e-ff4f-4d8a-b75c-35ca7a6515b8 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:11.245471 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 414/826] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:11.245471 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c886b-5351-4abb-89ec-4839a17188a9 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:11.326298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ab51e-ff4f-4d8a-b75c-35ca7a6515b8 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:11.372806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7467e5e-a9a2-4609-8cd6-aeb9e8f96b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:11.373462 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7467e5e-a9a2-4609-8cd6-aeb9e8f96b08 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:11.373742 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7467e5e-a9a2-4609-8cd6-aeb9e8f96b08 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:11.374015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7467e5e-a9a2-4609-8cd6-aeb9e8f96b08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:11.374772 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7467e5e-a9a2-4609-8cd6-aeb9e8f96b08 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:11.376150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7467e5e-a9a2-4609-8cd6-aeb9e8f96b08 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:11.376563 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c886b-5351-4abb-89ec-4839a17188a9 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:11.402722 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 413/827] 10.222.0.35 () {60 vars in 1144 bytes} [Fri May 13 21:39:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:39:11.404982 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7467e5e-a9a2-4609-8cd6-aeb9e8f96b08 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:11.421778 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961cb2b-14ce-4f00-8b19-d0ce94a4b562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:11.422478 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961cb2b-14ce-4f00-8b19-d0ce94a4b562 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:11.422797 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961cb2b-14ce-4f00-8b19-d0ce94a4b562 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:11.423075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961cb2b-14ce-4f00-8b19-d0ce94a4b562 None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:11.423930 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961cb2b-14ce-4f00-8b19-d0ce94a4b562 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:11.424240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961cb2b-14ce-4f00-8b19-d0ce94a4b562 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:11.425708 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961cb2b-14ce-4f00-8b19-d0ce94a4b562 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:11.444677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 415/828] 10.222.0.35 () {56 vars in 1133 bytes} [Fri May 13 21:39:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:11.451837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961cb2b-14ce-4f00-8b19-d0ce94a4b562 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:11.584925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961cb2b-14ce-4f00-8b19-d0ce94a4b562 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:11.636829 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a612f-82d4-4413-8de5-2bf10423fe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:11.637880 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a612f-82d4-4413-8de5-2bf10423fe6c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:11.639604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a612f-82d4-4413-8de5-2bf10423fe6c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:11.640137 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a612f-82d4-4413-8de5-2bf10423fe6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:11.646577 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7467e5e-a9a2-4609-8cd6-aeb9e8f96b08 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:11.691794 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed45c84-0ed0-4560-9ed6-ff778839fb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:11.692646 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed45c84-0ed0-4560-9ed6-ff778839fb89 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:11.692929 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed45c84-0ed0-4560-9ed6-ff778839fb89 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:11.693206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed45c84-0ed0-4560-9ed6-ff778839fb89 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:11.693919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed45c84-0ed0-4560-9ed6-ff778839fb89 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:11.695373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed45c84-0ed0-4560-9ed6-ff778839fb89 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-3ed45c84-0ed0-4560-9ed6-ff778839fb89 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:11.707790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:11.708808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:11.715185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 414/829] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:39:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:11.720939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed45c84-0ed0-4560-9ed6-ff778839fb89 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:11.734489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a612f-82d4-4413-8de5-2bf10423fe6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:11.736281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a612f-82d4-4413-8de5-2bf10423fe6c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:11.828717 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed45c84-0ed0-4560-9ed6-ff778839fb89 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:11.846383 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 416/830] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:39:11 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:39:11.847958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a612f-82d4-4413-8de5-2bf10423fe6c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:11.864961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc43f1-923b-4492-b059-18ea20a8a63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:11.865623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc43f1-923b-4492-b059-18ea20a8a63d None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:11.865906 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc43f1-923b-4492-b059-18ea20a8a63d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:11.866184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc43f1-923b-4492-b059-18ea20a8a63d None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:11.866918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc43f1-923b-4492-b059-18ea20a8a63d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:11.867228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc43f1-923b-4492-b059-18ea20a8a63d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:11.868618 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc43f1-923b-4492-b059-18ea20a8a63d None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:11.882209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 415/831] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:39:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:11.884589 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc43f1-923b-4492-b059-18ea20a8a63d None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:11.975931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a612f-82d4-4413-8de5-2bf10423fe6c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:12.000546 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc43f1-923b-4492-b059-18ea20a8a63d None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:12.011408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b33fc-2041-448f-80e0-dd1a05cd1774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:12.012444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b33fc-2041-448f-80e0-dd1a05cd1774 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:12.012881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b33fc-2041-448f-80e0-dd1a05cd1774 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:12.013334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b33fc-2041-448f-80e0-dd1a05cd1774 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:12.023841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b33fc-2041-448f-80e0-dd1a05cd1774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:12.025472 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b33fc-2041-448f-80e0-dd1a05cd1774 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:12.043032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df21bc2-b558-4df8-b1be-6c0d24204aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-812b33fc-2041-448f-80e0-dd1a05cd1774 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:12.043702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:12.045248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:12.046144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:12.046144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:12.046144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:12.046144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:12.046363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df21bc2-b558-4df8-b1be-6c0d24204aa3 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:12.046363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df21bc2-b558-4df8-b1be-6c0d24204aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:12.046363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df21bc2-b558-4df8-b1be-6c0d24204aa3 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:12.046363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df21bc2-b558-4df8-b1be-6c0d24204aa3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:12.046617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df21bc2-b558-4df8-b1be-6c0d24204aa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:12.048501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df21bc2-b558-4df8-b1be-6c0d24204aa3 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:12.053114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 417/832] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:12.053997 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 416/833] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:39:11 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:12.054829 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b33fc-2041-448f-80e0-dd1a05cd1774 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:12.174183 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b33fc-2041-448f-80e0-dd1a05cd1774 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:12.212156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7d016-8e87-4dcd-b181-7303f037012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:12.213192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7d016-8e87-4dcd-b181-7303f037012c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:12.213673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7d016-8e87-4dcd-b181-7303f037012c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:12.214141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7d016-8e87-4dcd-b181-7303f037012c None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:12.215153 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7d016-8e87-4dcd-b181-7303f037012c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:12.215693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7d016-8e87-4dcd-b181-7303f037012c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:12.218100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7d016-8e87-4dcd-b181-7303f037012c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:12.238927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 418/834] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:12.242297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df21bc2-b558-4df8-b1be-6c0d24204aa3 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:12.345274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df21bc2-b558-4df8-b1be-6c0d24204aa3 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:12.379184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766cfba-8ff3-40c9-aeb9-6ef44b37186e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:12.380269 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766cfba-8ff3-40c9-aeb9-6ef44b37186e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:12.380757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766cfba-8ff3-40c9-aeb9-6ef44b37186e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:12.381218 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766cfba-8ff3-40c9-aeb9-6ef44b37186e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:12.388158 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766cfba-8ff3-40c9-aeb9-6ef44b37186e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:12.389860 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766cfba-8ff3-40c9-aeb9-6ef44b37186e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-d766cfba-8ff3-40c9-aeb9-6ef44b37186e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:12.405026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:12.405909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:12.406828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:12.406828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:12.406828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:12.406828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:12.406828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:12.406828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:12.406828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:12.406828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:12.413247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 417/835] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:12.414746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7d016-8e87-4dcd-b181-7303f037012c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:12.433314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005dd7-43c3-4347-9744-1ddebe30aeac None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:12.433826 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005dd7-43c3-4347-9744-1ddebe30aeac None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:12.434288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005dd7-43c3-4347-9744-1ddebe30aeac None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:12.436550 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 418/836] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:39:12.441881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fddd32-c2df-4dbf-8d4e-572e6b941891 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:12.442329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fddd32-c2df-4dbf-8d4e-572e6b941891 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:12.442790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fddd32-c2df-4dbf-8d4e-572e6b941891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:12.464482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d5fddd32-c2df-4dbf-8d4e-572e6b941891 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:12.467096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d5fddd32-c2df-4dbf-8d4e-572e6b941891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:12.520718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7d016-8e87-4dcd-b181-7303f037012c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:12.551859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 419/837] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:12.563740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9527460-fcf2-4055-9141-8dfa3216b115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:12.564611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9527460-fcf2-4055-9141-8dfa3216b115 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:12.565108 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9527460-fcf2-4055-9141-8dfa3216b115 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:12.565536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9527460-fcf2-4055-9141-8dfa3216b115 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:12.566474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9527460-fcf2-4055-9141-8dfa3216b115 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:12.566994 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9527460-fcf2-4055-9141-8dfa3216b115 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:12.568651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9527460-fcf2-4055-9141-8dfa3216b115 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:12.578767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fddd32-c2df-4dbf-8d4e-572e6b941891 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:12.588187 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 419/838] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:12.595562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9527460-fcf2-4055-9141-8dfa3216b115 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:12.698284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fddd32-c2df-4dbf-8d4e-572e6b941891 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:12.719356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9527460-fcf2-4055-9141-8dfa3216b115 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:12.733220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94f8b0-0eec-4890-92a6-9838fc443ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:12.734100 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94f8b0-0eec-4890-92a6-9838fc443ae6 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:12.734455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94f8b0-0eec-4890-92a6-9838fc443ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:12.734847 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94f8b0-0eec-4890-92a6-9838fc443ae6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:12.745569 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94f8b0-0eec-4890-92a6-9838fc443ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-1c94f8b0-0eec-4890-92a6-9838fc443ae6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:12.753581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:12.754502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:12.755605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:12.755605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:12.755605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:12.755605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:12.755605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:12.755605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:12.755605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:12.755605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:12.755605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:12.755605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:12.755605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:12.755605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:12.759759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5337a-c5ab-4149-a8eb-acbc53ce501b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:12.759939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 420/839] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:39:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:12.760774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94f8b0-0eec-4890-92a6-9838fc443ae6 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:12.760978 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5337a-c5ab-4149-a8eb-acbc53ce501b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:12.761504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5337a-c5ab-4149-a8eb-acbc53ce501b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:12.761958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5337a-c5ab-4149-a8eb-acbc53ce501b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:12.763576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5337a-c5ab-4149-a8eb-acbc53ce501b None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=316ef43c6c304bd3a27e085bf4da8a5e) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:12.764091 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5337a-c5ab-4149-a8eb-acbc53ce501b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:12.765706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5337a-c5ab-4149-a8eb-acbc53ce501b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:12.779438 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 420/840] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:39:12 2022] GET /identity/v3/role_assignments?user.id=316ef43c6c304bd3a27e085bf4da8a5e&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:12.876234 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94f8b0-0eec-4890-92a6-9838fc443ae6 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:12.913206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1e992-f541-4325-8747-2059e807df89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:12.913882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1e992-f541-4325-8747-2059e807df89 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:12.914164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1e992-f541-4325-8747-2059e807df89 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:12.914441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1e992-f541-4325-8747-2059e807df89 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:12.915227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1e992-f541-4325-8747-2059e807df89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:12.915534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1e992-f541-4325-8747-2059e807df89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:12.916936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1e992-f541-4325-8747-2059e807df89 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:12.926794 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 421/841] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:39:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:12.932374 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5337a-c5ab-4149-a8eb-acbc53ce501b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:13.038222 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5337a-c5ab-4149-a8eb-acbc53ce501b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:13.072178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e53d4-65ee-4265-b036-e3f38621796b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:13.073405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e53d4-65ee-4265-b036-e3f38621796b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:13.073886 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e53d4-65ee-4265-b036-e3f38621796b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:13.074381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e53d4-65ee-4265-b036-e3f38621796b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:13.084562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e53d4-65ee-4265-b036-e3f38621796b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:13.086225 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e53d4-65ee-4265-b036-e3f38621796b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-ff1e53d4-65ee-4265-b036-e3f38621796b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:13.104285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:13.105189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:13.106074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:13.106074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:13.106074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:13.106074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:13.108857 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 421/842] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:13.114576 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1e992-f541-4325-8747-2059e807df89 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:13.220475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1e992-f541-4325-8747-2059e807df89 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:13.254335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ad1f0-949b-4cff-843e-31e56e491abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:13.255120 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ad1f0-949b-4cff-843e-31e56e491abd None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:13.255405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ad1f0-949b-4cff-843e-31e56e491abd None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:13.255683 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ad1f0-949b-4cff-843e-31e56e491abd None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:13.256412 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ad1f0-949b-4cff-843e-31e56e491abd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:13.256719 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ad1f0-949b-4cff-843e-31e56e491abd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:13.258135 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ad1f0-949b-4cff-843e-31e56e491abd None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:13.272567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 422/843] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:13.278055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e53d4-65ee-4265-b036-e3f38621796b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:13.344497 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493f7a3-4e67-4898-8f70-ea5932b2f62c None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:13.344796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493f7a3-4e67-4898-8f70-ea5932b2f62c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:13.344980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493f7a3-4e67-4898-8f70-ea5932b2f62c None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:13.346463 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 423/844] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:39:13.351686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6e254-0206-4a5b-8c7b-8447c103f1fc None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:13.352219 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6e254-0206-4a5b-8c7b-8447c103f1fc None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:13.352693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6e254-0206-4a5b-8c7b-8447c103f1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:13.370010 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-06f6e254-0206-4a5b-8c7b-8447c103f1fc None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:13.371567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-06f6e254-0206-4a5b-8c7b-8447c103f1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:13.390806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e53d4-65ee-4265-b036-e3f38621796b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:13.434895 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771f46b-88b3-497c-ae14-5af276bfefb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:13.435886 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771f46b-88b3-497c-ae14-5af276bfefb3 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:13.436178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771f46b-88b3-497c-ae14-5af276bfefb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:13.436455 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771f46b-88b3-497c-ae14-5af276bfefb3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:13.443941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771f46b-88b3-497c-ae14-5af276bfefb3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:13.444468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 424/845] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:13.445393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771f46b-88b3-497c-ae14-5af276bfefb3 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:13.462377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6e254-0206-4a5b-8c7b-8447c103f1fc None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-5771f46b-88b3-497c-ae14-5af276bfefb3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:13.462769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:13.463664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:13.464585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:13.464585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:13.464585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:13.464585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:13.464585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:13.464585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:13.464585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:13.464585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:13.464585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 21:39:13.464585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:13.464585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 422/846] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:39:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:13.471182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771f46b-88b3-497c-ae14-5af276bfefb3 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:13.569295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771f46b-88b3-497c-ae14-5af276bfefb3 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:13.594043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6e254-0206-4a5b-8c7b-8447c103f1fc None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:13.604128 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c299b8d-8eba-4b77-8339-deef6d65b6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:13.605072 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c299b8d-8eba-4b77-8339-deef6d65b6e4 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:13.605578 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c299b8d-8eba-4b77-8339-deef6d65b6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:13.606050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c299b8d-8eba-4b77-8339-deef6d65b6e4 None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:13.607146 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c299b8d-8eba-4b77-8339-deef6d65b6e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:13.607666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c299b8d-8eba-4b77-8339-deef6d65b6e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:13.609295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c299b8d-8eba-4b77-8339-deef6d65b6e4 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:13.627818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 423/847] 10.222.0.35 () {56 vars in 1135 bytes} [Fri May 13 21:39:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:13.633771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c299b8d-8eba-4b77-8339-deef6d65b6e4 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:13.641687 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcbd39-3fb4-40e9-baf7-aa19d1e13a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:13.642814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcbd39-3fb4-40e9-baf7-aa19d1e13a6e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:13.643310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcbd39-3fb4-40e9-baf7-aa19d1e13a6e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:13.643777 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcbd39-3fb4-40e9-baf7-aa19d1e13a6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:13.654047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcbd39-3fb4-40e9-baf7-aa19d1e13a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-23dcbd39-3fb4-40e9-baf7-aa19d1e13a6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:13.662015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:13.662967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:13.663802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:13.663802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:13.663802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:13.663802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:13.663802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:13.663802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:13.663802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:13.663802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:13.663802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:13.663802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:13.663802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:39:13.663802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:13.671094 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 425/848] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:39:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:13.672180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcbd39-3fb4-40e9-baf7-aa19d1e13a6e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:13.751501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c299b8d-8eba-4b77-8339-deef6d65b6e4 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:13.773191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcbd39-3fb4-40e9-baf7-aa19d1e13a6e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:13.794576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea6f89-2016-41a4-9e1c-8b5b2344b23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:13.795463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea6f89-2016-41a4-9e1c-8b5b2344b23a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:13.795755 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea6f89-2016-41a4-9e1c-8b5b2344b23a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:13.796033 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea6f89-2016-41a4-9e1c-8b5b2344b23a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:13.806766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f250c-e52c-49da-ad8f-3c9338de3bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:13.807667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f250c-e52c-49da-ad8f-3c9338de3bd7 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:13.808121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f250c-e52c-49da-ad8f-3c9338de3bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:13.808566 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f250c-e52c-49da-ad8f-3c9338de3bd7 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:13.809539 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f250c-e52c-49da-ad8f-3c9338de3bd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:13.810546 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f250c-e52c-49da-ad8f-3c9338de3bd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:13.812110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f250c-e52c-49da-ad8f-3c9338de3bd7 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:13.833596 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 426/849] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:39:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:13.834755 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f250c-e52c-49da-ad8f-3c9338de3bd7 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:13.885298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea6f89-2016-41a4-9e1c-8b5b2344b23a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:13.887175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea6f89-2016-41a4-9e1c-8b5b2344b23a None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:13.933640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f250c-e52c-49da-ad8f-3c9338de3bd7 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:13.978922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5edb21-d2a4-493b-b8b9-2f951eaf7eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:13.980007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5edb21-d2a4-493b-b8b9-2f951eaf7eff None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:13.980297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5edb21-d2a4-493b-b8b9-2f951eaf7eff None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:13.980577 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5edb21-d2a4-493b-b8b9-2f951eaf7eff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:13.995795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5edb21-d2a4-493b-b8b9-2f951eaf7eff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:13.997259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5edb21-d2a4-493b-b8b9-2f951eaf7eff None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:13.999386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 424/850] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:39:13 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:39:14.005128 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea6f89-2016-41a4-9e1c-8b5b2344b23a None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-7e5edb21-d2a4-493b-b8b9-2f951eaf7eff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:14.019674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:14.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:14.021608 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:14.021608 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:14.021608 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:14.021608 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:14.028269 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 427/851] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:14.029414 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5edb21-d2a4-493b-b8b9-2f951eaf7eff None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:14.114593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea6f89-2016-41a4-9e1c-8b5b2344b23a None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:14.153533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa4909-ad4e-4033-9691-27668052763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:14.153858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5edb21-d2a4-493b-b8b9-2f951eaf7eff None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:14.155528 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa4909-ad4e-4033-9691-27668052763f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:14.156034 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa4909-ad4e-4033-9691-27668052763f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:14.156532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa4909-ad4e-4033-9691-27668052763f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:14.168600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa4909-ad4e-4033-9691-27668052763f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:14.170877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa4909-ad4e-4033-9691-27668052763f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-47aa4909-ad4e-4033-9691-27668052763f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:14.193375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:14.194333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:14.195272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:14.195272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:14.195272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:14.195272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:14.195272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 425/852] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:14.196696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9cfda-9414-47ff-ac0c-65457cfcd4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:14.197351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9cfda-9414-47ff-ac0c-65457cfcd4ab None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:14.197633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9cfda-9414-47ff-ac0c-65457cfcd4ab None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:14.197910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9cfda-9414-47ff-ac0c-65457cfcd4ab None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:14.198626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9cfda-9414-47ff-ac0c-65457cfcd4ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:14.198963 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9cfda-9414-47ff-ac0c-65457cfcd4ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:14.200359 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9cfda-9414-47ff-ac0c-65457cfcd4ab None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:14.208286 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa4909-ad4e-4033-9691-27668052763f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:14.216044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 428/853] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:14.222147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9cfda-9414-47ff-ac0c-65457cfcd4ab None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:14.314493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa4909-ad4e-4033-9691-27668052763f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:14.323783 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9cfda-9414-47ff-ac0c-65457cfcd4ab None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:14.349711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5ea6c-5d69-4c97-8e13-a22e93cadcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:14.350381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5ea6c-5d69-4c97-8e13-a22e93cadcbe None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:14.350873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5ea6c-5d69-4c97-8e13-a22e93cadcbe None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:14.351156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5ea6c-5d69-4c97-8e13-a22e93cadcbe None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:14.352207 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5ea6c-5d69-4c97-8e13-a22e93cadcbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:14.352514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5ea6c-5d69-4c97-8e13-a22e93cadcbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:14.354317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5ea6c-5d69-4c97-8e13-a22e93cadcbe None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:14.360473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6c83b-8e79-4249-b3cc-740bc78af5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:14.361534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6c83b-8e79-4249-b3cc-740bc78af5de None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:14.362017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6c83b-8e79-4249-b3cc-740bc78af5de None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:14.362511 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6c83b-8e79-4249-b3cc-740bc78af5de None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:14.370745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 426/854] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:14.373956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5ea6c-5d69-4c97-8e13-a22e93cadcbe None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:14.374733 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6c83b-8e79-4249-b3cc-740bc78af5de None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:14.376274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6c83b-8e79-4249-b3cc-740bc78af5de None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-10d6c83b-8e79-4249-b3cc-740bc78af5de None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:14.399228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:14.400235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:14.401171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:14.401171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:14.401171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:14.401171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:14.401171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:14.401171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:14.401171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:14.401171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:14.401171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 21:39:14.401171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:14.403366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 429/855] 10.222.0.35 () {56 vars in 1117 bytes} [Fri May 13 21:39:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:14.405324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ea845-6dff-4fec-aad0-b3643df017a2 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:14.405550 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ea845-6dff-4fec-aad0-b3643df017a2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:14.405822 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ea845-6dff-4fec-aad0-b3643df017a2 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:14.407315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 430/856] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:14.411845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ea845-6dff-4fec-aad0-b3643df017a2 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:14.478664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5ea6c-5d69-4c97-8e13-a22e93cadcbe None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:14.509214 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ea845-6dff-4fec-aad0-b3643df017a2 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:14.515424 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebb299-d0b9-4b36-a63a-dda0a2262b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:14.516729 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebb299-d0b9-4b36-a63a-dda0a2262b8c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:14.517028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebb299-d0b9-4b36-a63a-dda0a2262b8c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:14.517307 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebb299-d0b9-4b36-a63a-dda0a2262b8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:14.526214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebb299-d0b9-4b36-a63a-dda0a2262b8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:14.527692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebb299-d0b9-4b36-a63a-dda0a2262b8c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:14.542058 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd319e6-4070-4a9f-a241-93fb2e701b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-06ebb299-d0b9-4b36-a63a-dda0a2262b8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:14.542746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:14.543592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:14.545076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:14.545076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:14.545076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:14.545076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:14.545076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:14.545076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:14.545076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:14.545076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:14.545507 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd319e6-4070-4a9f-a241-93fb2e701b3b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:14.545507 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd319e6-4070-4a9f-a241-93fb2e701b3b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:14.545507 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd319e6-4070-4a9f-a241-93fb2e701b3b None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:14.545685 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 427/857] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:14.546243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214ce1d-f802-4bb7-aced-5ae92ca69155 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:14.546407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214ce1d-f802-4bb7-aced-5ae92ca69155 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:14.546684 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214ce1d-f802-4bb7-aced-5ae92ca69155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:14.549182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd319e6-4070-4a9f-a241-93fb2e701b3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:14.549723 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd319e6-4070-4a9f-a241-93fb2e701b3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:14.551263 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd319e6-4070-4a9f-a241-93fb2e701b3b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:14.563585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-c214ce1d-f802-4bb7-aced-5ae92ca69155 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:14.564748 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c214ce1d-f802-4bb7-aced-5ae92ca69155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:14.573431 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 431/858] 10.222.0.35 () {56 vars in 1149 bytes} [Fri May 13 21:39:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:14.573431 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd319e6-4070-4a9f-a241-93fb2e701b3b None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:14.630464 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 428/859] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:14.631593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214ce1d-f802-4bb7-aced-5ae92ca69155 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:14.682304 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd319e6-4070-4a9f-a241-93fb2e701b3b None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:14.715206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c6f52-c03f-4248-8ae8-94396d318ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:14.715855 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c6f52-c03f-4248-8ae8-94396d318ccd None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:14.716698 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c6f52-c03f-4248-8ae8-94396d318ccd None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:14.716972 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c6f52-c03f-4248-8ae8-94396d318ccd None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:14.717748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c6f52-c03f-4248-8ae8-94396d318ccd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:14.718046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c6f52-c03f-4248-8ae8-94396d318ccd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:14.719500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c6f52-c03f-4248-8ae8-94396d318ccd None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:14.730665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 432/860] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:14.731467 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c6f52-c03f-4248-8ae8-94396d318ccd None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:14.745373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214ce1d-f802-4bb7-aced-5ae92ca69155 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:14.783273 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed22caf-e61d-442d-8883-485c4172e957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:14.784088 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed22caf-e61d-442d-8883-485c4172e957 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:14.784374 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed22caf-e61d-442d-8883-485c4172e957 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:14.784652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed22caf-e61d-442d-8883-485c4172e957 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:14.827302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c6f52-c03f-4248-8ae8-94396d318ccd None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:14.849853 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed22caf-e61d-442d-8883-485c4172e957 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:14.851510 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed22caf-e61d-442d-8883-485c4172e957 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:14.862111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e1cc6-eef7-4b84-a587-a57ec4d7e2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:14.866664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e1cc6-eef7-4b84-a587-a57ec4d7e2c8 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:14.866664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e1cc6-eef7-4b84-a587-a57ec4d7e2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:14.866664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e1cc6-eef7-4b84-a587-a57ec4d7e2c8 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:14.866664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e1cc6-eef7-4b84-a587-a57ec4d7e2c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:14.866664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e1cc6-eef7-4b84-a587-a57ec4d7e2c8 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:14.884851 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 433/861] 10.222.0.35 () {60 vars in 1141 bytes} [Fri May 13 21:39:14 2022] POST /identity/v3/services => generated 251 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:39:14.885663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e1cc6-eef7-4b84-a587-a57ec4d7e2c8 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:14.965435 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 429/862] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:39:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:39:14.971378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed22caf-e61d-442d-8883-485c4172e957 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:15.043439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e1cc6-eef7-4b84-a587-a57ec4d7e2c8 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:15.082097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52d6a6-207f-4091-b3da-c37fafcf6748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:15.083211 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52d6a6-207f-4091-b3da-c37fafcf6748 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:15.083507 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52d6a6-207f-4091-b3da-c37fafcf6748 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:15.083783 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52d6a6-207f-4091-b3da-c37fafcf6748 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:15.084618 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52d6a6-207f-4091-b3da-c37fafcf6748 None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=4f9e07ed0a084c82b946aaf585de16fd) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:15.084930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52d6a6-207f-4091-b3da-c37fafcf6748 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:15.086438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52d6a6-207f-4091-b3da-c37fafcf6748 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:15.094133 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 434/863] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:39:14 2022] GET /identity/v3/role_assignments?user.id=4f9e07ed0a084c82b946aaf585de16fd&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:15.100054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86cc90-2cca-4349-822b-6f90520bf288 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:15.100356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86cc90-2cca-4349-822b-6f90520bf288 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:15.100638 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86cc90-2cca-4349-822b-6f90520bf288 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:15.102070 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 435/864] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:39:15.108632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604d24c-4796-4ba2-ae57-73ff731203c1 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:15.108951 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed22caf-e61d-442d-8883-485c4172e957 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:15.109085 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604d24c-4796-4ba2-ae57-73ff731203c1 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:15.109162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604d24c-4796-4ba2-ae57-73ff731203c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:15.132103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7604d24c-4796-4ba2-ae57-73ff731203c1 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:15.133602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7604d24c-4796-4ba2-ae57-73ff731203c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:15.162209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15542f0f-6a57-4f75-8022-d2d2248c4dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:15.164041 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542f0f-6a57-4f75-8022-d2d2248c4dec None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:15.164796 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542f0f-6a57-4f75-8022-d2d2248c4dec None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:15.165263 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542f0f-6a57-4f75-8022-d2d2248c4dec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:15.180754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15542f0f-6a57-4f75-8022-d2d2248c4dec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:15.183216 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15542f0f-6a57-4f75-8022-d2d2248c4dec None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-15542f0f-6a57-4f75-8022-d2d2248c4dec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:15.208640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:15.209823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:15.210907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:15.210907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:15.210907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:15.210907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:15.212263 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 430/865] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:15.218129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15542f0f-6a57-4f75-8022-d2d2248c4dec None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:15.221974 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 436/866] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:15 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:15.245615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604d24c-4796-4ba2-ae57-73ff731203c1 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:15.335860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15542f0f-6a57-4f75-8022-d2d2248c4dec None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:15.357913 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604d24c-4796-4ba2-ae57-73ff731203c1 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:15.371618 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb073d-88e4-4d31-bac3-a946f6f7908d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:15.372389 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb073d-88e4-4d31-bac3-a946f6f7908d None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:15.372680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb073d-88e4-4d31-bac3-a946f6f7908d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:15.372961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb073d-88e4-4d31-bac3-a946f6f7908d None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:15.373837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb073d-88e4-4d31-bac3-a946f6f7908d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:15.374047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb073d-88e4-4d31-bac3-a946f6f7908d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:15.375459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb073d-88e4-4d31-bac3-a946f6f7908d None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:15.387708 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 431/867] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:15.393209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb073d-88e4-4d31-bac3-a946f6f7908d None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:15.394578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79cf140-463f-49e9-8648-0c8d43a9a829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:15.395602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cf140-463f-49e9-8648-0c8d43a9a829 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:15.396047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cf140-463f-49e9-8648-0c8d43a9a829 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:15.396475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cf140-463f-49e9-8648-0c8d43a9a829 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:15.397273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79cf140-463f-49e9-8648-0c8d43a9a829 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:15.398827 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79cf140-463f-49e9-8648-0c8d43a9a829 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-e79cf140-463f-49e9-8648-0c8d43a9a829 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:15.411796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:15.412975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 21:39:15.413918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:15.416294 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 437/868] 10.222.0.35 () {56 vars in 1132 bytes} [Fri May 13 21:39:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:15.424023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79cf140-463f-49e9-8648-0c8d43a9a829 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:15.497155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb073d-88e4-4d31-bac3-a946f6f7908d None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:15.525944 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79cf140-463f-49e9-8648-0c8d43a9a829 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:15.532258 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd4ef9-7515-4b20-9f89-391884895af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:15.533053 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd4ef9-7515-4b20-9f89-391884895af1 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:15.533331 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd4ef9-7515-4b20-9f89-391884895af1 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:15.533607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd4ef9-7515-4b20-9f89-391884895af1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:15.540568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd4ef9-7515-4b20-9f89-391884895af1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:15.542015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd4ef9-7515-4b20-9f89-391884895af1 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-94dd4ef9-7515-4b20-9f89-391884895af1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:15.556767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:15.557679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:15.558740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:15.558740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:15.558740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:15.558740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:15.558740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:15.558740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:15.558740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:15.558740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:15.558740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 432/869] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:15.564674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd4ef9-7515-4b20-9f89-391884895af1 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:15.569313 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67323620-c21b-452b-98dc-a2c7a4c82a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:15.570206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67323620-c21b-452b-98dc-a2c7a4c82a0b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:15.570490 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67323620-c21b-452b-98dc-a2c7a4c82a0b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:15.570785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67323620-c21b-452b-98dc-a2c7a4c82a0b None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:15.572397 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67323620-c21b-452b-98dc-a2c7a4c82a0b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:15.572716 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67323620-c21b-452b-98dc-a2c7a4c82a0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:15.574219 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67323620-c21b-452b-98dc-a2c7a4c82a0b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:15.580754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 438/870] 10.222.0.35 () {56 vars in 1126 bytes} [Fri May 13 21:39:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:15.586328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67323620-c21b-452b-98dc-a2c7a4c82a0b None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:15.671269 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd4ef9-7515-4b20-9f89-391884895af1 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:15.707989 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67323620-c21b-452b-98dc-a2c7a4c82a0b None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:15.711954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefbaf7-4e76-4653-84b8-5a0adf31ca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:15.712769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefbaf7-4e76-4653-84b8-5a0adf31ca2d None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:15.713151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefbaf7-4e76-4653-84b8-5a0adf31ca2d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:15.713527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefbaf7-4e76-4653-84b8-5a0adf31ca2d None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:15.714422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefbaf7-4e76-4653-84b8-5a0adf31ca2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:15.714865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefbaf7-4e76-4653-84b8-5a0adf31ca2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:15.717467 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefbaf7-4e76-4653-84b8-5a0adf31ca2d None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:15.732587 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 433/871] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:15.738013 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefbaf7-4e76-4653-84b8-5a0adf31ca2d None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:15.756727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e02d01-7940-46b1-b2ec-03928b2b6c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:15.757916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e02d01-7940-46b1-b2ec-03928b2b6c8a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:15.758721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e02d01-7940-46b1-b2ec-03928b2b6c8a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:15.759267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e02d01-7940-46b1-b2ec-03928b2b6c8a None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:15.760302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e02d01-7940-46b1-b2ec-03928b2b6c8a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:15.761472 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e02d01-7940-46b1-b2ec-03928b2b6c8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:15.764028 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e02d01-7940-46b1-b2ec-03928b2b6c8a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:15.779683 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 439/872] 10.222.0.35 () {56 vars in 1126 bytes} [Fri May 13 21:39:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:15.853492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefbaf7-4e76-4653-84b8-5a0adf31ca2d None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:15.891487 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398636ee-10c7-4944-b226-feffc9ea45e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:15.892156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398636ee-10c7-4944-b226-feffc9ea45e6 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:15.892437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398636ee-10c7-4944-b226-feffc9ea45e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:15.892712 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398636ee-10c7-4944-b226-feffc9ea45e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:15.893545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398636ee-10c7-4944-b226-feffc9ea45e6 None admin] RBAC: Adding query filter params (role.id=deb38f497be14e78ae21e8dceca3772f, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=a079ae1557d04831b73c03c96e2c7ad6) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:15.893854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398636ee-10c7-4944-b226-feffc9ea45e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:15.895338 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398636ee-10c7-4944-b226-feffc9ea45e6 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:15.903270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 434/873] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:39:15 2022] GET /identity/v3/role_assignments?user.id=a079ae1557d04831b73c03c96e2c7ad6&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=deb38f497be14e78ae21e8dceca3772f => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:17.216320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c96dd-7165-454f-8d82-a82b386b98b8 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:17.216831 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c96dd-7165-454f-8d82-a82b386b98b8 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:17.216910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c96dd-7165-454f-8d82-a82b386b98b8 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:17.218467 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 440/874] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:17.223751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b898e-f0e4-4d40-a41c-b5586e142bcb None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:17.224178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b898e-f0e4-4d40-a41c-b5586e142bcb None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:17.224345 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b898e-f0e4-4d40-a41c-b5586e142bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:17.243275 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-dc5b898e-f0e4-4d40-a41c-b5586e142bcb None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:17.244476 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-dc5b898e-f0e4-4d40-a41c-b5586e142bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:17.322479 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 435/875] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:17.383663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c96dd-7165-454f-8d82-a82b386b98b8 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:17.512907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c96dd-7165-454f-8d82-a82b386b98b8 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:17.532979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce2bd5-38d2-4181-9544-e7ebad9c5851 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:17.533482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce2bd5-38d2-4181-9544-e7ebad9c5851 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:17.533903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce2bd5-38d2-4181-9544-e7ebad9c5851 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:17.535467 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 436/876] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:17.543238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd116e-5a91-473b-9d5f-8bb52b404a33 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:17.543522 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd116e-5a91-473b-9d5f-8bb52b404a33 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:17.543797 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd116e-5a91-473b-9d5f-8bb52b404a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:17.555614 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad988dfa-73b2-41cc-af35-394f83ba14aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:17.556413 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad988dfa-73b2-41cc-af35-394f83ba14aa None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:17.556694 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad988dfa-73b2-41cc-af35-394f83ba14aa None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:17.556970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad988dfa-73b2-41cc-af35-394f83ba14aa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:17.557693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad988dfa-73b2-41cc-af35-394f83ba14aa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:17.559296 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad988dfa-73b2-41cc-af35-394f83ba14aa None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-ad988dfa-73b2-41cc-af35-394f83ba14aa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:17.581024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:17.582046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:17.582910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 441/877] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:39:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:17.583710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-fadd116e-5a91-473b-9d5f-8bb52b404a33 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:17.588507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-fadd116e-5a91-473b-9d5f-8bb52b404a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:17.591526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad988dfa-73b2-41cc-af35-394f83ba14aa None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:17.678732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 437/878] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:17.713594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd116e-5a91-473b-9d5f-8bb52b404a33 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:17.746764 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad988dfa-73b2-41cc-af35-394f83ba14aa None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:17.789664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f51556-831b-4216-aa4d-cbea69893b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:17.790248 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f51556-831b-4216-aa4d-cbea69893b14 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:17.790544 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f51556-831b-4216-aa4d-cbea69893b14 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:17.790851 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f51556-831b-4216-aa4d-cbea69893b14 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:17.792347 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f51556-831b-4216-aa4d-cbea69893b14 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:17.792662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f51556-831b-4216-aa4d-cbea69893b14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:17.794079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f51556-831b-4216-aa4d-cbea69893b14 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:17.799428 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 442/879] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:39:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:17.804996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f51556-831b-4216-aa4d-cbea69893b14 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:17.826753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd116e-5a91-473b-9d5f-8bb52b404a33 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:17.867055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630da9f0-51d4-407f-8f2c-49d2bedf6bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:17.868405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630da9f0-51d4-407f-8f2c-49d2bedf6bf7 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:17.868783 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630da9f0-51d4-407f-8f2c-49d2bedf6bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:17.869271 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630da9f0-51d4-407f-8f2c-49d2bedf6bf7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:17.870054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630da9f0-51d4-407f-8f2c-49d2bedf6bf7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:17.871986 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630da9f0-51d4-407f-8f2c-49d2bedf6bf7 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-630da9f0-51d4-407f-8f2c-49d2bedf6bf7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:17.888759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:17.890101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:17.891205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 438/880] 10.222.0.35 () {56 vars in 1126 bytes} [Fri May 13 21:39:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:17.895770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630da9f0-51d4-407f-8f2c-49d2bedf6bf7 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:17.921870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f51556-831b-4216-aa4d-cbea69893b14 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:17.956214 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b57a0-66ff-4b2f-a1d3-2f798d160097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:17.956871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b57a0-66ff-4b2f-a1d3-2f798d160097 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:17.957200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b57a0-66ff-4b2f-a1d3-2f798d160097 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:17.957477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b57a0-66ff-4b2f-a1d3-2f798d160097 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:17.958178 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b57a0-66ff-4b2f-a1d3-2f798d160097 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:17.958483 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b57a0-66ff-4b2f-a1d3-2f798d160097 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:17.959942 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b57a0-66ff-4b2f-a1d3-2f798d160097 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:17.965385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 443/881] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:39:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:17.971322 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b57a0-66ff-4b2f-a1d3-2f798d160097 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:18.024404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630da9f0-51d4-407f-8f2c-49d2bedf6bf7 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:18.063455 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2c0ba-d8da-46e7-b174-906323e4d971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:18.064239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2c0ba-d8da-46e7-b174-906323e4d971 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:18.064668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2c0ba-d8da-46e7-b174-906323e4d971 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:18.065046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2c0ba-d8da-46e7-b174-906323e4d971 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:18.065882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2c0ba-d8da-46e7-b174-906323e4d971 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:18.066352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2c0ba-d8da-46e7-b174-906323e4d971 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:18.067909 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2c0ba-d8da-46e7-b174-906323e4d971 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:18.079070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 439/882] 10.222.0.35 () {56 vars in 1122 bytes} [Fri May 13 21:39:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:18.081009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2c0ba-d8da-46e7-b174-906323e4d971 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:18.093045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b57a0-66ff-4b2f-a1d3-2f798d160097 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:18.136409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0ccb1-8197-42e2-9f87-f561bbe7ddff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:18.137341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0ccb1-8197-42e2-9f87-f561bbe7ddff None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:18.137626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0ccb1-8197-42e2-9f87-f561bbe7ddff None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:18.138469 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0ccb1-8197-42e2-9f87-f561bbe7ddff None admin] PATH_INFO: `/v3/endpoints` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:18.139445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0ccb1-8197-42e2-9f87-f561bbe7ddff None admin] RBAC: Adding query filter params (interface=public, service_id=7db37b5c1fe3490d92186b38a22e4efd, region_id=RegionOne) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:18.140078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0ccb1-8197-42e2-9f87-f561bbe7ddff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:18.141722 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0ccb1-8197-42e2-9f87-f561bbe7ddff None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:18.151122 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 444/883] 10.222.0.35 () {56 vars in 1251 bytes} [Fri May 13 21:39:17 2022] GET /identity/v3/endpoints?service_id=7db37b5c1fe3490d92186b38a22e4efd&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:18.156626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0ccb1-8197-42e2-9f87-f561bbe7ddff None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:18.201327 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2c0ba-d8da-46e7-b174-906323e4d971 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:18.239088 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ab237-db67-4327-8dea-4491e84b2360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:18.240551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab237-db67-4327-8dea-4491e84b2360 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:18.240990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab237-db67-4327-8dea-4491e84b2360 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:18.241422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab237-db67-4327-8dea-4491e84b2360 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:18.242274 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ab237-db67-4327-8dea-4491e84b2360 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:18.242758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ab237-db67-4327-8dea-4491e84b2360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:18.244386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ab237-db67-4327-8dea-4491e84b2360 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:18.256083 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 440/884] 10.222.0.35 () {56 vars in 1122 bytes} [Fri May 13 21:39:18 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:18.256083 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a3490-d215-47f2-a725-52b972cfb18d None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:18.256083 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a3490-d215-47f2-a725-52b972cfb18d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:18.256083 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a3490-d215-47f2-a725-52b972cfb18d None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:18.256083 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 441/885] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:18.260132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe890af-3943-4a4a-92eb-1b9bc13655e7 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:18.260424 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe890af-3943-4a4a-92eb-1b9bc13655e7 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:18.260695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe890af-3943-4a4a-92eb-1b9bc13655e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:18.273950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0ccb1-8197-42e2-9f87-f561bbe7ddff None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:18.279208 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-cfe890af-3943-4a4a-92eb-1b9bc13655e7 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:18.280428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-cfe890af-3943-4a4a-92eb-1b9bc13655e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:18.310872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a151a67-c75e-4c53-8d5a-aceff4137d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:18.311548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151a67-c75e-4c53-8d5a-aceff4137d6b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:18.311832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151a67-c75e-4c53-8d5a-aceff4137d6b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:18.312109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151a67-c75e-4c53-8d5a-aceff4137d6b None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:18.312797 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a151a67-c75e-4c53-8d5a-aceff4137d6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:18.314279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a151a67-c75e-4c53-8d5a-aceff4137d6b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:18.327027 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 445/886] 10.222.0.35 () {56 vars in 1087 bytes} [Fri May 13 21:39:18 2022] GET /identity/v3/services => generated 1011 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:39:18.362907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 442/887] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:18 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:18.382585 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a151a67-c75e-4c53-8d5a-aceff4137d6b None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:18.389640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbcd3c-d52a-4fb5-9ed6-d7a15c0aef34 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:18.389921 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbcd3c-d52a-4fb5-9ed6-d7a15c0aef34 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:18.390199 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbcd3c-d52a-4fb5-9ed6-d7a15c0aef34 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:18.396535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 443/888] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:39:18.398038 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f42f0-2ce9-4a17-90c1-f7020c93314a None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:18.398326 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f42f0-2ce9-4a17-90c1-f7020c93314a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:18.398603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f42f0-2ce9-4a17-90c1-f7020c93314a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:18.417270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5a3f42f0-2ce9-4a17-90c1-f7020c93314a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:18.418497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5a3f42f0-2ce9-4a17-90c1-f7020c93314a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:18.506171 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 444/889] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:18 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:18.513219 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a151a67-c75e-4c53-8d5a-aceff4137d6b None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:18.530045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f42f0-2ce9-4a17-90c1-f7020c93314a None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:18.554124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a8a67-4327-4bd8-96bb-239d2d979520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:18.554957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a8a67-4327-4bd8-96bb-239d2d979520 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:18.555242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a8a67-4327-4bd8-96bb-239d2d979520 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:18.555547 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a8a67-4327-4bd8-96bb-239d2d979520 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:18.556291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a8a67-4327-4bd8-96bb-239d2d979520 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:18.557666 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a8a67-4327-4bd8-96bb-239d2d979520 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:18.582930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 446/890] 10.222.0.35 () {60 vars in 1141 bytes} [Fri May 13 21:39:18 2022] POST /identity/v3/services => generated 269 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:39:18.662006 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f42f0-2ce9-4a17-90c1-f7020c93314a None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:18.714802 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04fbbf-9383-44fc-9a64-30e029978086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:18.715637 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04fbbf-9383-44fc-9a64-30e029978086 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:18.716028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04fbbf-9383-44fc-9a64-30e029978086 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:18.716306 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04fbbf-9383-44fc-9a64-30e029978086 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:18.728073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04fbbf-9383-44fc-9a64-30e029978086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-6b04fbbf-9383-44fc-9a64-30e029978086 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:18.737189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:18.738180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:18.739079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:18.739079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:18.739079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:18.739079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:18.739079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:18.739079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:18.739079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:18.739079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:18.739079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:18.739079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:18.739079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 21:39:18.739079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:18.739079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 445/891] 10.222.0.35 () {56 vars in 1120 bytes} [Fri May 13 21:39:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:18.748437 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a8a67-4327-4bd8-96bb-239d2d979520 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:18.854542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a8a67-4327-4bd8-96bb-239d2d979520 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:18.888046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cc7a0-4009-4ba1-8d7f-b6bbae73509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:18.888706 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cc7a0-4009-4ba1-8d7f-b6bbae73509e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:18.888984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cc7a0-4009-4ba1-8d7f-b6bbae73509e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:18.889256 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cc7a0-4009-4ba1-8d7f-b6bbae73509e None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:18.889987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cc7a0-4009-4ba1-8d7f-b6bbae73509e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:18.890569 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cc7a0-4009-4ba1-8d7f-b6bbae73509e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:18.892068 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cc7a0-4009-4ba1-8d7f-b6bbae73509e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:18.900261 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 447/892] 10.222.0.35 () {56 vars in 1115 bytes} [Fri May 13 21:39:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:18.906034 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04fbbf-9383-44fc-9a64-30e029978086 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:19.023998 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04fbbf-9383-44fc-9a64-30e029978086 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:19.059051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d98804-355a-4159-bca0-e5a0777dd9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:19.060270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d98804-355a-4159-bca0-e5a0777dd9c7 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:19.060776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d98804-355a-4159-bca0-e5a0777dd9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:19.061275 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d98804-355a-4159-bca0-e5a0777dd9c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:19.072622 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d98804-355a-4159-bca0-e5a0777dd9c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:19.074332 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d98804-355a-4159-bca0-e5a0777dd9c7 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-54d98804-355a-4159-bca0-e5a0777dd9c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:19.092798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:19.093718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:19.094594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:19.094594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:19.094594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:19.094594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:19.096861 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 446/893] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:19.102232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cc7a0-4009-4ba1-8d7f-b6bbae73509e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:19.206361 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cc7a0-4009-4ba1-8d7f-b6bbae73509e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:19.239402 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d204e-3397-4b52-bb86-bf44943b97a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:19.240060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d204e-3397-4b52-bb86-bf44943b97a6 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:19.240340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d204e-3397-4b52-bb86-bf44943b97a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:19.240614 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d204e-3397-4b52-bb86-bf44943b97a6 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:19.241327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d204e-3397-4b52-bb86-bf44943b97a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:19.241634 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d204e-3397-4b52-bb86-bf44943b97a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:19.243034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d204e-3397-4b52-bb86-bf44943b97a6 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:19.256529 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 448/894] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:19.261658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d98804-355a-4159-bca0-e5a0777dd9c7 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:19.365597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d98804-355a-4159-bca0-e5a0777dd9c7 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:19.399896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456eea1-2445-41e0-b273-44dbc1c8b86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:19.400705 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456eea1-2445-41e0-b273-44dbc1c8b86e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:19.400990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456eea1-2445-41e0-b273-44dbc1c8b86e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:19.401269 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456eea1-2445-41e0-b273-44dbc1c8b86e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:19.408946 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456eea1-2445-41e0-b273-44dbc1c8b86e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:19.410346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456eea1-2445-41e0-b273-44dbc1c8b86e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-2456eea1-2445-41e0-b273-44dbc1c8b86e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:19.425740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:19.426764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:19.427736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:19.427736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:19.427736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:19.427736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:19.427736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:19.427736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:19.427736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:19.427736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:19.427736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 21:39:19.427736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:19.427736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 447/895] 10.222.0.35 () {56 vars in 1117 bytes} [Fri May 13 21:39:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:19.433237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d204e-3397-4b52-bb86-bf44943b97a6 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:19.535490 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d204e-3397-4b52-bb86-bf44943b97a6 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:19.570385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e8c07-ff15-428e-8c57-7d9a55a05d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:19.571107 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e8c07-ff15-428e-8c57-7d9a55a05d3a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:19.571392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e8c07-ff15-428e-8c57-7d9a55a05d3a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:19.571671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e8c07-ff15-428e-8c57-7d9a55a05d3a None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:19.572530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e8c07-ff15-428e-8c57-7d9a55a05d3a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:19.572843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e8c07-ff15-428e-8c57-7d9a55a05d3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:19.574338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e8c07-ff15-428e-8c57-7d9a55a05d3a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:19.594276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 449/896] 10.222.0.35 () {56 vars in 1149 bytes} [Fri May 13 21:39:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:19.599804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456eea1-2445-41e0-b273-44dbc1c8b86e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:19.703098 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456eea1-2445-41e0-b273-44dbc1c8b86e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:19.736573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27dfdb-7f92-48ca-a56a-3e4695a6d835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:19.737215 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27dfdb-7f92-48ca-a56a-3e4695a6d835 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:19.737494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27dfdb-7f92-48ca-a56a-3e4695a6d835 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:19.737769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27dfdb-7f92-48ca-a56a-3e4695a6d835 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:19.809195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27dfdb-7f92-48ca-a56a-3e4695a6d835 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:19.810731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27dfdb-7f92-48ca-a56a-3e4695a6d835 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:19.909778 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 448/897] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:39:19 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:39:19.915150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e8c07-ff15-428e-8c57-7d9a55a05d3a None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:20.015779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e8c07-ff15-428e-8c57-7d9a55a05d3a None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:20.049928 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada1d7e-8206-4a98-bf0f-44258da5dd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:20.050738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada1d7e-8206-4a98-bf0f-44258da5dd06 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:20.051028 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada1d7e-8206-4a98-bf0f-44258da5dd06 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:20.051307 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada1d7e-8206-4a98-bf0f-44258da5dd06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:20.061412 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada1d7e-8206-4a98-bf0f-44258da5dd06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:20.062905 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada1d7e-8206-4a98-bf0f-44258da5dd06 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-0ada1d7e-8206-4a98-bf0f-44258da5dd06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:20.080967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:20.081868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:20.082848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:20.082848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:20.082848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:20.082848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:20.082848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 450/898] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:20.087992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27dfdb-7f92-48ca-a56a-3e4695a6d835 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:20.189572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27dfdb-7f92-48ca-a56a-3e4695a6d835 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:20.224866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c655e-e077-4e8a-84a3-56cbb3b75fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:20.225534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c655e-e077-4e8a-84a3-56cbb3b75fc0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:20.225818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c655e-e077-4e8a-84a3-56cbb3b75fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:20.226094 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c655e-e077-4e8a-84a3-56cbb3b75fc0 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:20.226859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c655e-e077-4e8a-84a3-56cbb3b75fc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:20.227169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c655e-e077-4e8a-84a3-56cbb3b75fc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:20.228546 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c655e-e077-4e8a-84a3-56cbb3b75fc0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:20.240571 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 449/899] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:20.245992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada1d7e-8206-4a98-bf0f-44258da5dd06 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:20.348105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada1d7e-8206-4a98-bf0f-44258da5dd06 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:20.383059 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013afac4-7e3a-4fbc-9b21-4897971b4037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:20.383868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013afac4-7e3a-4fbc-9b21-4897971b4037 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:20.384153 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013afac4-7e3a-4fbc-9b21-4897971b4037 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:20.384432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013afac4-7e3a-4fbc-9b21-4897971b4037 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:20.392912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013afac4-7e3a-4fbc-9b21-4897971b4037 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:20.394421 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013afac4-7e3a-4fbc-9b21-4897971b4037 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-013afac4-7e3a-4fbc-9b21-4897971b4037 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:20.409479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:20.410444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:20.411352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:20.411352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:20.411352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:20.411352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:20.411352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:20.411352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:20.411352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:20.411352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:20.411352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 451/900] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:20.416789 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c655e-e077-4e8a-84a3-56cbb3b75fc0 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:20.520623 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c655e-e077-4e8a-84a3-56cbb3b75fc0 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:20.557862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff4713-e250-4618-b622-7159456a73d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:20.558526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff4713-e250-4618-b622-7159456a73d1 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:20.558843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff4713-e250-4618-b622-7159456a73d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:20.559123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff4713-e250-4618-b622-7159456a73d1 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:20.560650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff4713-e250-4618-b622-7159456a73d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:20.560962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff4713-e250-4618-b622-7159456a73d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:20.562401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff4713-e250-4618-b622-7159456a73d1 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:20.575879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 450/901] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:20.581729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013afac4-7e3a-4fbc-9b21-4897971b4037 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:20.588590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0ca40-b1ad-420a-ba33-a5cabccc0efa None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:20.588590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0ca40-b1ad-420a-ba33-a5cabccc0efa None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:20.588590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0ca40-b1ad-420a-ba33-a5cabccc0efa None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:20.589348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 451/902] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:20.594536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4d3da-6aac-425d-a968-6f240e05e397 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:20.595110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4d3da-6aac-425d-a968-6f240e05e397 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:20.595589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4d3da-6aac-425d-a968-6f240e05e397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:20.612037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-60d4d3da-6aac-425d-a968-6f240e05e397 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:20.613539 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-60d4d3da-6aac-425d-a968-6f240e05e397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:20.686791 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 452/903] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:20.697778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013afac4-7e3a-4fbc-9b21-4897971b4037 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:20.706762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4d3da-6aac-425d-a968-6f240e05e397 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:20.736183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4964e70-e2b1-4f9d-88ae-1474673ffcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:20.736910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964e70-e2b1-4f9d-88ae-1474673ffcd2 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:20.737202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964e70-e2b1-4f9d-88ae-1474673ffcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:20.737484 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964e70-e2b1-4f9d-88ae-1474673ffcd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:20.738342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4964e70-e2b1-4f9d-88ae-1474673ffcd2 None admin] RBAC: Adding query filter params (role.id=708710710e3e44dca3a6d11cbc2b6dd6, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=a079ae1557d04831b73c03c96e2c7ad6) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:20.738661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4964e70-e2b1-4f9d-88ae-1474673ffcd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:20.740259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4964e70-e2b1-4f9d-88ae-1474673ffcd2 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:20.746513 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 452/904] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:39:20 2022] GET /identity/v3/role_assignments?user.id=a079ae1557d04831b73c03c96e2c7ad6&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=708710710e3e44dca3a6d11cbc2b6dd6 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:20.748907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d5e90-c25d-4d6f-a301-03a9435e2e2a None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:20.749423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d5e90-c25d-4d6f-a301-03a9435e2e2a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:20.750578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d5e90-c25d-4d6f-a301-03a9435e2e2a None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:20.753500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 453/905] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:20.759702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0689c-3fd8-43d6-a76e-b7e7e572724b None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:20.760393 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0689c-3fd8-43d6-a76e-b7e7e572724b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:20.761092 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0689c-3fd8-43d6-a76e-b7e7e572724b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:20.785499 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-8bf0689c-3fd8-43d6-a76e-b7e7e572724b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:20.787143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-8bf0689c-3fd8-43d6-a76e-b7e7e572724b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:20.815787 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4d3da-6aac-425d-a968-6f240e05e397 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:20.853129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d61354-5983-4da8-945c-a6893bd6ecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:20.854010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d61354-5983-4da8-945c-a6893bd6ecd3 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:20.854122 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d61354-5983-4da8-945c-a6893bd6ecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:20.854420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d61354-5983-4da8-945c-a6893bd6ecd3 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:20.855222 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d61354-5983-4da8-945c-a6893bd6ecd3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:20.857224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d61354-5983-4da8-945c-a6893bd6ecd3 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:20.882815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 453/906] 10.222.0.35 () {60 vars in 1141 bytes} [Fri May 13 21:39:20 2022] POST /identity/v3/services => generated 241 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:39:20.894337 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 454/907] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:20.917703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d61354-5983-4da8-945c-a6893bd6ecd3 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:20.940637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16624a8-1e16-4a63-b1a1-15d6e5032900 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:20.940991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16624a8-1e16-4a63-b1a1-15d6e5032900 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:20.941197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16624a8-1e16-4a63-b1a1-15d6e5032900 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:20.946854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 455/908] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:39:20.949009 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41cae1-890d-4104-b07d-cd135d194183 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:20.949290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41cae1-890d-4104-b07d-cd135d194183 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:20.949561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41cae1-890d-4104-b07d-cd135d194183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:20.972153 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-1a41cae1-890d-4104-b07d-cd135d194183 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:20.973727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-1a41cae1-890d-4104-b07d-cd135d194183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:21.060818 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 456/909] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:21.067852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d61354-5983-4da8-945c-a6893bd6ecd3 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:21.083792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41cae1-890d-4104-b07d-cd135d194183 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:21.107505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3937f-c701-4058-8d5f-5db564b13a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:21.108702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3937f-c701-4058-8d5f-5db564b13a85 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:21.109188 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3937f-c701-4058-8d5f-5db564b13a85 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:21.109651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3937f-c701-4058-8d5f-5db564b13a85 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:21.110510 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3937f-c701-4058-8d5f-5db564b13a85 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:21.112082 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3937f-c701-4058-8d5f-5db564b13a85 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-94a3937f-c701-4058-8d5f-5db564b13a85 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:21.126229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:21.127674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:21.128502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:21.128502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:21.128502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:21.128502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:21.128502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:21.128502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:21.128502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:21.128502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:21.128502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:21.128502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:21.128502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:21.128502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:21.128502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 21:39:21.128502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:21.129654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 454/910] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:39:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:21.135663 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3937f-c701-4058-8d5f-5db564b13a85 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:21.198625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41cae1-890d-4104-b07d-cd135d194183 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:21.233229 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14b61c-637d-4e95-b5b9-1ea585505f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:21.234140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14b61c-637d-4e95-b5b9-1ea585505f66 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:21.234497 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14b61c-637d-4e95-b5b9-1ea585505f66 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:21.234890 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14b61c-637d-4e95-b5b9-1ea585505f66 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:21.235620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14b61c-637d-4e95-b5b9-1ea585505f66 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:21.237175 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14b61c-637d-4e95-b5b9-1ea585505f66 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:21.243104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3937f-c701-4058-8d5f-5db564b13a85 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-ea14b61c-637d-4e95-b5b9-1ea585505f66 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:21.250233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:21.251268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 21:39:21.252377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:21.256764 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 457/911] 10.222.0.35 () {56 vars in 1132 bytes} [Fri May 13 21:39:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:21.257985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14b61c-637d-4e95-b5b9-1ea585505f66 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:21.278890 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a363b31-3f0d-4f75-83ba-23790e44b3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:21.279554 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a363b31-3f0d-4f75-83ba-23790e44b3aa None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:21.279833 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a363b31-3f0d-4f75-83ba-23790e44b3aa None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:21.280111 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a363b31-3f0d-4f75-83ba-23790e44b3aa None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:21.280884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a363b31-3f0d-4f75-83ba-23790e44b3aa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:21.281188 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a363b31-3f0d-4f75-83ba-23790e44b3aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:21.282630 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a363b31-3f0d-4f75-83ba-23790e44b3aa None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:21.288036 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 455/912] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:39:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:21.293447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a363b31-3f0d-4f75-83ba-23790e44b3aa None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:21.361773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14b61c-637d-4e95-b5b9-1ea585505f66 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:21.396702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51469f50-772c-4d04-8c37-ee857289f354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:21.397325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51469f50-772c-4d04-8c37-ee857289f354 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:21.397610 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51469f50-772c-4d04-8c37-ee857289f354 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:21.397887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51469f50-772c-4d04-8c37-ee857289f354 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:21.398592 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51469f50-772c-4d04-8c37-ee857289f354 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:21.399056 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51469f50-772c-4d04-8c37-ee857289f354 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:21.400393 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51469f50-772c-4d04-8c37-ee857289f354 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:21.410100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a363b31-3f0d-4f75-83ba-23790e44b3aa None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:21.410286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 458/913] 10.222.0.35 () {56 vars in 1126 bytes} [Fri May 13 21:39:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:21.411437 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51469f50-772c-4d04-8c37-ee857289f354 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:21.445803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caccd9-fa71-40ef-96f1-3a08b8773c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:21.447217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caccd9-fa71-40ef-96f1-3a08b8773c4c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:21.447763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caccd9-fa71-40ef-96f1-3a08b8773c4c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:21.448300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caccd9-fa71-40ef-96f1-3a08b8773c4c None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:21.449191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3caccd9-fa71-40ef-96f1-3a08b8773c4c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:21.449688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3caccd9-fa71-40ef-96f1-3a08b8773c4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:21.451906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3caccd9-fa71-40ef-96f1-3a08b8773c4c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:21.461876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 456/914] 10.222.0.35 () {56 vars in 1114 bytes} [Fri May 13 21:39:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:21.463071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caccd9-fa71-40ef-96f1-3a08b8773c4c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:21.512439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51469f50-772c-4d04-8c37-ee857289f354 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:21.546795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00702a-7e41-4bc9-b596-6206e1d5edbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:21.547465 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00702a-7e41-4bc9-b596-6206e1d5edbe None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:21.547750 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00702a-7e41-4bc9-b596-6206e1d5edbe None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:21.548027 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00702a-7e41-4bc9-b596-6206e1d5edbe None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:21.548731 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00702a-7e41-4bc9-b596-6206e1d5edbe None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:21.549039 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00702a-7e41-4bc9-b596-6206e1d5edbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:21.550422 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00702a-7e41-4bc9-b596-6206e1d5edbe None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:21.561043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 459/915] 10.222.0.35 () {56 vars in 1126 bytes} [Fri May 13 21:39:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:21.562121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00702a-7e41-4bc9-b596-6206e1d5edbe None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:21.570613 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caccd9-fa71-40ef-96f1-3a08b8773c4c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:21.604615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32602f-946a-4730-a87e-ef044580b21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:21.605652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32602f-946a-4730-a87e-ef044580b21b None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:21.606183 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32602f-946a-4730-a87e-ef044580b21b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:21.607249 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32602f-946a-4730-a87e-ef044580b21b None admin] PATH_INFO: `/v3/endpoints` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:21.608294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32602f-946a-4730-a87e-ef044580b21b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:21.609910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32602f-946a-4730-a87e-ef044580b21b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:21.643600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 457/916] 10.222.0.35 () {60 vars in 1144 bytes} [Fri May 13 21:39:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:39:21.678015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00702a-7e41-4bc9-b596-6206e1d5edbe None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:21.726601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c3b89-fb6f-415e-8f48-4c2637d2ad63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:21.726601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c3b89-fb6f-415e-8f48-4c2637d2ad63 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:21.726601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c3b89-fb6f-415e-8f48-4c2637d2ad63 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:21.726601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c3b89-fb6f-415e-8f48-4c2637d2ad63 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:21.727042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c3b89-fb6f-415e-8f48-4c2637d2ad63 None admin] RBAC: Adding query filter params (interface=public, service_id=f969fb30d0fc42b782033e9b19d7689d, region_id=RegionOne) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:21.727042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c3b89-fb6f-415e-8f48-4c2637d2ad63 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:21.728553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c3b89-fb6f-415e-8f48-4c2637d2ad63 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:21.740466 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 460/917] 10.222.0.35 () {56 vars in 1251 bytes} [Fri May 13 21:39:21 2022] GET /identity/v3/endpoints?service_id=f969fb30d0fc42b782033e9b19d7689d&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:21.742037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32602f-946a-4730-a87e-ef044580b21b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:21.856608 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32602f-946a-4730-a87e-ef044580b21b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:21.893920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984cebcb-b003-4cc1-a868-4eb84aa95f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:21.894595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984cebcb-b003-4cc1-a868-4eb84aa95f02 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:21.894924 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984cebcb-b003-4cc1-a868-4eb84aa95f02 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:21.895203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984cebcb-b003-4cc1-a868-4eb84aa95f02 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:21.895889 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984cebcb-b003-4cc1-a868-4eb84aa95f02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:21.897274 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984cebcb-b003-4cc1-a868-4eb84aa95f02 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:21.906778 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 458/918] 10.222.0.35 () {56 vars in 1087 bytes} [Fri May 13 21:39:21 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:39:23.099301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504179c1-5fef-48a6-875d-4cf77ff15fb5 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:23.099827 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504179c1-5fef-48a6-875d-4cf77ff15fb5 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:23.099939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504179c1-5fef-48a6-875d-4cf77ff15fb5 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:23.101479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 461/919] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:23.106859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e5309-3063-4edd-bb71-d1f04d338a2f None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:23.107253 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e5309-3063-4edd-bb71-d1f04d338a2f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:23.107433 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e5309-3063-4edd-bb71-d1f04d338a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:23.123991 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ac2e5309-3063-4edd-bb71-d1f04d338a2f None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:23.125177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ac2e5309-3063-4edd-bb71-d1f04d338a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:23.194495 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 459/920] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:23.217570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504179c1-5fef-48a6-875d-4cf77ff15fb5 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:23.225726 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d9f3b-4181-420d-bc87-88f11be59b49 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:23.226268 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d9f3b-4181-420d-bc87-88f11be59b49 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:23.226759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d9f3b-4181-420d-bc87-88f11be59b49 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:23.228556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 460/921] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:39:23.234092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ee04a-431f-42a5-923d-19d66116d61f None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:23.234562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ee04a-431f-42a5-923d-19d66116d61f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:23.235026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ee04a-431f-42a5-923d-19d66116d61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:23.256417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-092ee04a-431f-42a5-923d-19d66116d61f None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:23.257832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-092ee04a-431f-42a5-923d-19d66116d61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:23.329575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504179c1-5fef-48a6-875d-4cf77ff15fb5 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:23.342727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 461/922] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:23.365177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ee04a-431f-42a5-923d-19d66116d61f None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:23.369569 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95b062-64fe-4d32-8bab-74e158cf4c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:23.370373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95b062-64fe-4d32-8bab-74e158cf4c6d None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:23.370655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95b062-64fe-4d32-8bab-74e158cf4c6d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:23.370976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95b062-64fe-4d32-8bab-74e158cf4c6d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:23.382214 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95b062-64fe-4d32-8bab-74e158cf4c6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-3e95b062-64fe-4d32-8bab-74e158cf4c6d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:23.390334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:23.391310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:23.392137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:23.392137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:23.392137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:23.392137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:23.392137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:23.392137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:23.392137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:23.392137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:23.392137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:23.392137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:23.392137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 21:39:23.392137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:23.392137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 462/923] 10.222.0.35 () {56 vars in 1120 bytes} [Fri May 13 21:39:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:23.397603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95b062-64fe-4d32-8bab-74e158cf4c6d None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:23.489472 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ee04a-431f-42a5-923d-19d66116d61f None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:23.509934 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95b062-64fe-4d32-8bab-74e158cf4c6d None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:23.526156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096c86a-b70f-42b2-890b-451837580910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:23.527289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096c86a-b70f-42b2-890b-451837580910 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:23.527289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096c86a-b70f-42b2-890b-451837580910 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:23.527587 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096c86a-b70f-42b2-890b-451837580910 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:23.528752 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096c86a-b70f-42b2-890b-451837580910 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:23.530135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096c86a-b70f-42b2-890b-451837580910 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-0096c86a-b70f-42b2-890b-451837580910 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:23.544860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:23.545904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:23.546844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:23.546844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:23.546844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:23.546844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:23.546844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:23.546844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:23.546844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:23.546844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:23.546844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:23.546844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:23.546844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:23.546844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:23.546844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:23.546844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:23.551955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 462/924] 10.222.0.35 () {56 vars in 1126 bytes} [Fri May 13 21:39:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:23.552928 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096c86a-b70f-42b2-890b-451837580910 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:23.554867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a11f9-beb6-4688-9e79-e965621dab9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:23.556020 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a11f9-beb6-4688-9e79-e965621dab9e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:23.557449 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a11f9-beb6-4688-9e79-e965621dab9e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:23.557949 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a11f9-beb6-4688-9e79-e965621dab9e None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:23.559224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a11f9-beb6-4688-9e79-e965621dab9e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:23.559928 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a11f9-beb6-4688-9e79-e965621dab9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:23.561521 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a11f9-beb6-4688-9e79-e965621dab9e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:23.573321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 463/925] 10.222.0.35 () {56 vars in 1115 bytes} [Fri May 13 21:39:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:23.580091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a11f9-beb6-4688-9e79-e965621dab9e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:23.661639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096c86a-b70f-42b2-890b-451837580910 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:23.695680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18996318-4a0c-478f-ad70-70755016d2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:23.696488 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18996318-4a0c-478f-ad70-70755016d2f9 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:23.696769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18996318-4a0c-478f-ad70-70755016d2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:23.697045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18996318-4a0c-478f-ad70-70755016d2f9 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:23.697746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18996318-4a0c-478f-ad70-70755016d2f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:23.698051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18996318-4a0c-478f-ad70-70755016d2f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:23.700723 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18996318-4a0c-478f-ad70-70755016d2f9 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:23.710864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 463/926] 10.222.0.35 () {56 vars in 1122 bytes} [Fri May 13 21:39:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:23.713226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a11f9-beb6-4688-9e79-e965621dab9e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:23.715500 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18996318-4a0c-478f-ad70-70755016d2f9 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:23.755146 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f652810-e8f4-4d2d-8855-d719ef08e062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:23.756007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f652810-e8f4-4d2d-8855-d719ef08e062 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:23.756293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f652810-e8f4-4d2d-8855-d719ef08e062 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:23.756570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f652810-e8f4-4d2d-8855-d719ef08e062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:23.770425 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f652810-e8f4-4d2d-8855-d719ef08e062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:23.772109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f652810-e8f4-4d2d-8855-d719ef08e062 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-7f652810-e8f4-4d2d-8855-d719ef08e062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:23.795684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:23.801002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:23.803143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:23.803143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:23.803143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:23.803143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:23.803143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 464/927] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:23.803794 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f652810-e8f4-4d2d-8855-d719ef08e062 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:23.844357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18996318-4a0c-478f-ad70-70755016d2f9 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:23.878192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af440c4c-18df-4156-9a71-59f6c330990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:23.879300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af440c4c-18df-4156-9a71-59f6c330990c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:23.879300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af440c4c-18df-4156-9a71-59f6c330990c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:23.879502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af440c4c-18df-4156-9a71-59f6c330990c None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:23.880116 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af440c4c-18df-4156-9a71-59f6c330990c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:23.880423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af440c4c-18df-4156-9a71-59f6c330990c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:23.881801 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af440c4c-18df-4156-9a71-59f6c330990c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:23.892250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 464/928] 10.222.0.35 () {56 vars in 1122 bytes} [Fri May 13 21:39:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:23.893364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af440c4c-18df-4156-9a71-59f6c330990c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:23.922653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f652810-e8f4-4d2d-8855-d719ef08e062 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:23.964455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9c80b-c03d-409d-a3bd-878a99a2a907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:23.965154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9c80b-c03d-409d-a3bd-878a99a2a907 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:23.965601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9c80b-c03d-409d-a3bd-878a99a2a907 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:23.966581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9c80b-c03d-409d-a3bd-878a99a2a907 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:23.967460 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9c80b-c03d-409d-a3bd-878a99a2a907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:23.967775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9c80b-c03d-409d-a3bd-878a99a2a907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:23.969799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9c80b-c03d-409d-a3bd-878a99a2a907 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:23.984572 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 465/929] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:23.985909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e6103-b663-4bc6-8b02-3d780ddfd81a None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:23.986199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e6103-b663-4bc6-8b02-3d780ddfd81a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:23.986478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e6103-b663-4bc6-8b02-3d780ddfd81a None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:23.988082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 466/930] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:23.993716 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c9129-d98d-4305-9007-733e6e0bf47b None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:23.994007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c9129-d98d-4305-9007-733e6e0bf47b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:23.994285 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c9129-d98d-4305-9007-733e6e0bf47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:24.011673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-119c9129-d98d-4305-9007-733e6e0bf47b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:24.013308 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af440c4c-18df-4156-9a71-59f6c330990c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:24.013420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-119c9129-d98d-4305-9007-733e6e0bf47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:24.051725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8dc6d-8dfc-45d5-8618-230ddb08ffe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:24.052744 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8dc6d-8dfc-45d5-8618-230ddb08ffe9 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:24.053223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8dc6d-8dfc-45d5-8618-230ddb08ffe9 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:24.053691 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8dc6d-8dfc-45d5-8618-230ddb08ffe9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:24.055104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8dc6d-8dfc-45d5-8618-230ddb08ffe9 None admin] RBAC: Adding query filter params (interface=public, service_id=a46e47859a1441daa0c4dc7279aa1f4f, region_id=RegionOne) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:24.055658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8dc6d-8dfc-45d5-8618-230ddb08ffe9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:24.057608 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8dc6d-8dfc-45d5-8618-230ddb08ffe9 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:24.065582 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 465/931] 10.222.0.35 () {56 vars in 1251 bytes} [Fri May 13 21:39:23 2022] GET /identity/v3/endpoints?service_id=a46e47859a1441daa0c4dc7279aa1f4f&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:24.066573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8dc6d-8dfc-45d5-8618-230ddb08ffe9 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:24.084286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 467/932] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:24.087613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c9129-d98d-4305-9007-733e6e0bf47b None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:24.169074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8dc6d-8dfc-45d5-8618-230ddb08ffe9 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:24.188701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c9129-d98d-4305-9007-733e6e0bf47b None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:24.203923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753085fd-d092-4450-97ea-45e0c78cd9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:24.204727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753085fd-d092-4450-97ea-45e0c78cd9be None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:24.205007 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753085fd-d092-4450-97ea-45e0c78cd9be None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:24.205281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753085fd-d092-4450-97ea-45e0c78cd9be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:24.213458 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753085fd-d092-4450-97ea-45e0c78cd9be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:24.214932 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753085fd-d092-4450-97ea-45e0c78cd9be None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:24.222425 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8c66d-a8e3-448a-b0bb-8388fd524952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:24.223118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8c66d-a8e3-448a-b0bb-8388fd524952 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:24.223403 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8c66d-a8e3-448a-b0bb-8388fd524952 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:24.223676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8c66d-a8e3-448a-b0bb-8388fd524952 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:24.224350 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc8c66d-a8e3-448a-b0bb-8388fd524952 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:24.225694 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc8c66d-a8e3-448a-b0bb-8388fd524952 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-753085fd-d092-4450-97ea-45e0c78cd9be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:24.228809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:24.229891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:24.231022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:24.231022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:24.231022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:24.231022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:24.231022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:24.231022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:24.231022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:24.231022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:24.231022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 21:39:24.231022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:24.231022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 466/933] 10.222.0.35 () {56 vars in 1117 bytes} [Fri May 13 21:39:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:24.231022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753085fd-d092-4450-97ea-45e0c78cd9be None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:24.235029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 468/934] 10.222.0.35 () {56 vars in 1087 bytes} [Fri May 13 21:39:24 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:39:24.236275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb0b9f-4a27-44cf-a1d4-cd01323e2d21 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:24.236561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb0b9f-4a27-44cf-a1d4-cd01323e2d21 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:24.236837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb0b9f-4a27-44cf-a1d4-cd01323e2d21 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:24.238216 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 469/935] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:24.242757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb0b9f-4a27-44cf-a1d4-cd01323e2d21 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:24.347492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753085fd-d092-4450-97ea-45e0c78cd9be None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:24.348275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb0b9f-4a27-44cf-a1d4-cd01323e2d21 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:24.383196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19a37c-7325-45cf-bf10-9e12cb52d8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:24.383863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19a37c-7325-45cf-bf10-9e12cb52d8c5 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:24.384149 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19a37c-7325-45cf-bf10-9e12cb52d8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:24.384415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19a37c-7325-45cf-bf10-9e12cb52d8c5 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:24.385118 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19a37c-7325-45cf-bf10-9e12cb52d8c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:24.386587 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19a37c-7325-45cf-bf10-9e12cb52d8c5 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:24.388405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c0bc7-1802-453d-8caf-c18d5b4855f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:24.388821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: WARNING keystone.server.flask.common [None req-8a19a37c-7325-45cf-bf10-9e12cb52d8c5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 13 21:39:24.389126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c0bc7-1802-453d-8caf-c18d5b4855f6 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:24.389411 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c0bc7-1802-453d-8caf-c18d5b4855f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:24.389689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c0bc7-1802-453d-8caf-c18d5b4855f6 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:24.390538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c0bc7-1802-453d-8caf-c18d5b4855f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:24.390896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c0bc7-1802-453d-8caf-c18d5b4855f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:24.392361 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c0bc7-1802-453d-8caf-c18d5b4855f6 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:24.409300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.sql.core [None req-8a19a37c-7325-45cf-bf10-9e12cb52d8c5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 13 21:39:24.409300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 13 21:39:24.409300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [parameters: {'id': 'cd818fa8ecce43fc947d433d09b4a0e6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 13 21:39:24.409300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91431) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 13 21:39:24.413064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 470/936] 10.222.0.35 () {56 vars in 1149 bytes} [Fri May 13 21:39:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:24.419099 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44b418-e1d2-4c98-ac17-7529ab9c0d78 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:24.419099 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44b418-e1d2-4c98-ac17-7529ab9c0d78 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:24.419099 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44b418-e1d2-4c98-ac17-7529ab9c0d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:24.425605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-8a19a37c-7325-45cf-bf10-9e12cb52d8c5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cd818fa8ecce43fc947d433d09b4a0e6.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cd818fa8ecce43fc947d433d09b4a0e6. May 13 21:39:24.425605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:24.425605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 21:39:24.425605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 21:39:24.425605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 21:39:24.425605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 21:39:24.425605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 21:39:24.425605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application result = self._query(query) May 13 21:39:24.425605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 21:39:24.425605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application conn.query(q) May 13 21:39:24.425605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 21:39:24.425605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 21:39:24.425605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application result.read() May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 13 21:39:24.426437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return project_ref.to_dict() May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application next(self.gen) May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application yield resource May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application next(self.gen) May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self.session.rollback() May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self.force_reraise() May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise self.value May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application session.commit() May 13 21:39:24.427459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._prepare_impl() May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self.session.flush() May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._flush(objects) May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application compat.raise_( May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application flush_context.execute() May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 13 21:39:24.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rec.execute(self) May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application _emit_insert_statements( May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application c = connection._execute_20( May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ret = self._execute_context( May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 13 21:39:24.430516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application result = self._query(query) May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application conn.query(q) May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application result.read() May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 21:39:24.431514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [parameters: {'id': 'cd818fa8ecce43fc947d433d09b4a0e6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:24.432589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:24.433508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:24.437372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:24.437372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 13 21:39:24.437372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 13 21:39:24.437372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:24.437372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:24.437372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 13 21:39:24.437372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.Conflict( May 13 21:39:24.437372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cd818fa8ecce43fc947d433d09b4a0e6. May 13 21:39:24.437372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:24.437372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 467/937] 10.222.0.35 () {60 vars in 1140 bytes} [Fri May 13 21:39:24 2022] POST /identity/v3/projects => generated 262 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:24.437372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19a37c-7325-45cf-bf10-9e12cb52d8c5 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:24.442668 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-3f44b418-e1d2-4c98-ac17-7529ab9c0d78 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:24.443840 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3f44b418-e1d2-4c98-ac17-7529ab9c0d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:24.519624 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 471/938] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:24.520457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44b418-e1d2-4c98-ac17-7529ab9c0d78 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:24.542642 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19a37c-7325-45cf-bf10-9e12cb52d8c5 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:24.581536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057d6db-89c3-4c11-9c7e-b2cdac021e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:24.582640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057d6db-89c3-4c11-9c7e-b2cdac021e97 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:24.582986 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057d6db-89c3-4c11-9c7e-b2cdac021e97 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:24.583319 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057d6db-89c3-4c11-9c7e-b2cdac021e97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:24.593829 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057d6db-89c3-4c11-9c7e-b2cdac021e97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:24.595522 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057d6db-89c3-4c11-9c7e-b2cdac021e97 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-5057d6db-89c3-4c11-9c7e-b2cdac021e97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:24.612647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:24.613580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:24.614449 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:24.614449 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:24.614449 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:24.614449 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:24.614449 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 468/939] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:24.615555 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057d6db-89c3-4c11-9c7e-b2cdac021e97 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:24.640055 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44b418-e1d2-4c98-ac17-7529ab9c0d78 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:24.674834 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c91806-77a1-466b-8c46-f251261c3cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:24.675625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c91806-77a1-466b-8c46-f251261c3cfd None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:24.675903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c91806-77a1-466b-8c46-f251261c3cfd None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:24.676333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c91806-77a1-466b-8c46-f251261c3cfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:24.682624 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c91806-77a1-466b-8c46-f251261c3cfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:24.684095 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c91806-77a1-466b-8c46-f251261c3cfd None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-56c91806-77a1-466b-8c46-f251261c3cfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:24.697919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:24.698903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:24.699792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:24.699792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:24.699792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:24.699792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:24.699792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:24.699792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:24.699792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:24.699792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:24.699792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 472/940] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:24.699792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c91806-77a1-466b-8c46-f251261c3cfd None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:24.718433 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057d6db-89c3-4c11-9c7e-b2cdac021e97 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:24.755116 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234c309-cb87-405d-8766-168c3fe09ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:24.755929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234c309-cb87-405d-8766-168c3fe09ff8 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:24.756212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234c309-cb87-405d-8766-168c3fe09ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:24.756490 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234c309-cb87-405d-8766-168c3fe09ff8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:24.757143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234c309-cb87-405d-8766-168c3fe09ff8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:24.758515 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234c309-cb87-405d-8766-168c3fe09ff8 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-c234c309-cb87-405d-8766-168c3fe09ff8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:24.770675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:24.771813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:24.772750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 469/941] 10.222.0.35 () {56 vars in 1132 bytes} [Fri May 13 21:39:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:24.773901 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234c309-cb87-405d-8766-168c3fe09ff8 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:24.811636 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c91806-77a1-466b-8c46-f251261c3cfd None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:24.845713 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918abd9-b5c2-4df6-ae84-88ec4b91134c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:24.846373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918abd9-b5c2-4df6-ae84-88ec4b91134c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:24.846658 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918abd9-b5c2-4df6-ae84-88ec4b91134c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:24.846968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918abd9-b5c2-4df6-ae84-88ec4b91134c None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:24.847679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918abd9-b5c2-4df6-ae84-88ec4b91134c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:24.847987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918abd9-b5c2-4df6-ae84-88ec4b91134c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:24.849356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918abd9-b5c2-4df6-ae84-88ec4b91134c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:24.861490 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 473/942] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:24.862289 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918abd9-b5c2-4df6-ae84-88ec4b91134c None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:24.873089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234c309-cb87-405d-8766-168c3fe09ff8 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:24.906069 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07f41b-4dff-4ce1-8655-d910101701bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:24.906735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07f41b-4dff-4ce1-8655-d910101701bc None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:24.907026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07f41b-4dff-4ce1-8655-d910101701bc None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:24.907300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07f41b-4dff-4ce1-8655-d910101701bc None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:24.908047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07f41b-4dff-4ce1-8655-d910101701bc None admin] RBAC: Adding query filter params (name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:24.908356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07f41b-4dff-4ce1-8655-d910101701bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:24.909784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07f41b-4dff-4ce1-8655-d910101701bc None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:24.930322 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 470/943] 10.222.0.35 () {56 vars in 1112 bytes} [Fri May 13 21:39:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:24.931451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07f41b-4dff-4ce1-8655-d910101701bc None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:24.967832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918abd9-b5c2-4df6-ae84-88ec4b91134c None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:25.007542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9de4f1-2d8a-4f74-9ce2-c69fcfaaed40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:25.008678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9de4f1-2d8a-4f74-9ce2-c69fcfaaed40 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:25.009022 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9de4f1-2d8a-4f74-9ce2-c69fcfaaed40 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:25.009304 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9de4f1-2d8a-4f74-9ce2-c69fcfaaed40 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:25.010026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9de4f1-2d8a-4f74-9ce2-c69fcfaaed40 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:25.010333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9de4f1-2d8a-4f74-9ce2-c69fcfaaed40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:25.011873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9de4f1-2d8a-4f74-9ce2-c69fcfaaed40 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:25.023522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 474/944] 10.222.0.35 () {56 vars in 1126 bytes} [Fri May 13 21:39:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:25.024804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9de4f1-2d8a-4f74-9ce2-c69fcfaaed40 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:25.072371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07f41b-4dff-4ce1-8655-d910101701bc None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:25.109243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0c617-1549-46ba-998b-5d6c7f8b17f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:25.109825 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0c617-1549-46ba-998b-5d6c7f8b17f1 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:25.110105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0c617-1549-46ba-998b-5d6c7f8b17f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:25.110377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0c617-1549-46ba-998b-5d6c7f8b17f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:25.118009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c0c617-1549-46ba-998b-5d6c7f8b17f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:25.119679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c0c617-1549-46ba-998b-5d6c7f8b17f1 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-b2c0c617-1549-46ba-998b-5d6c7f8b17f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:25.134745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:25.135719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:25.136602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:25.136602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:25.136602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:25.136602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:25.136602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:25.136602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:25.136602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:25.136602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:25.136602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 471/945] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:25.142126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0c617-1549-46ba-998b-5d6c7f8b17f1 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:25.154664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9de4f1-2d8a-4f74-9ce2-c69fcfaaed40 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:25.189868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f093a4-2a46-42fc-a93b-bad23ba2de3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:25.190515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f093a4-2a46-42fc-a93b-bad23ba2de3b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:25.190833 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f093a4-2a46-42fc-a93b-bad23ba2de3b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:25.191113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f093a4-2a46-42fc-a93b-bad23ba2de3b None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:25.191817 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f093a4-2a46-42fc-a93b-bad23ba2de3b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:25.192118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f093a4-2a46-42fc-a93b-bad23ba2de3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:25.193503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f093a4-2a46-42fc-a93b-bad23ba2de3b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:25.198866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 475/946] 10.222.0.35 () {56 vars in 1126 bytes} [Fri May 13 21:39:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:25.204610 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f093a4-2a46-42fc-a93b-bad23ba2de3b None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:25.257666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0c617-1549-46ba-998b-5d6c7f8b17f1 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:25.292568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7212884-7240-41c9-9973-a102ebe94315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:25.293231 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7212884-7240-41c9-9973-a102ebe94315 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:25.293519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7212884-7240-41c9-9973-a102ebe94315 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:25.293952 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7212884-7240-41c9-9973-a102ebe94315 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:25.294770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7212884-7240-41c9-9973-a102ebe94315 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:25.295086 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7212884-7240-41c9-9973-a102ebe94315 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:25.296533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7212884-7240-41c9-9973-a102ebe94315 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:25.307875 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f093a4-2a46-42fc-a93b-bad23ba2de3b None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:25.308659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 472/947] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:25.314040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7212884-7240-41c9-9973-a102ebe94315 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:25.343437 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec465fb1-120d-487f-b65e-1e73a0912231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:25.344469 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec465fb1-120d-487f-b65e-1e73a0912231 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:25.344946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec465fb1-120d-487f-b65e-1e73a0912231 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:25.345415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec465fb1-120d-487f-b65e-1e73a0912231 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:25.346337 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec465fb1-120d-487f-b65e-1e73a0912231 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:25.347926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec465fb1-120d-487f-b65e-1e73a0912231 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:25.371079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 476/948] 10.222.0.35 () {60 vars in 1144 bytes} [Fri May 13 21:39:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:39:25.432827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7212884-7240-41c9-9973-a102ebe94315 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:25.480352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbf44d-838d-480b-b601-65079d3badf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:25.481698 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbf44d-838d-480b-b601-65079d3badf8 None admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:25.482181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbf44d-838d-480b-b601-65079d3badf8 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:25.482690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbf44d-838d-480b-b601-65079d3badf8 None admin] PATH_INFO: `/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/a079ae1557d04831b73c03c96e2c7ad6/roles/708710710e3e44dca3a6d11cbc2b6dd6` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:25.485606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbf44d-838d-480b-b601-65079d3badf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac834e84b2c84261ade311a6dfbbeb20, user_id=a079ae1557d04831b73c03c96e2c7ad6, role_id=708710710e3e44dca3a6d11cbc2b6dd6)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:25.487591 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbf44d-838d-480b-b601-65079d3badf8 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:25.503485 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 473/949] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:39:25 2022] PUT /identity/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/a079ae1557d04831b73c03c96e2c7ad6/roles/708710710e3e44dca3a6d11cbc2b6dd6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:39:26.512027 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fd890-44a5-4712-8950-3534e7a98e2d None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:26.512593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fd890-44a5-4712-8950-3534e7a98e2d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:26.512593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fd890-44a5-4712-8950-3534e7a98e2d None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:26.514206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 477/950] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:26.519747 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21285115-6507-4ab7-a461-b36b673a0cb4 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:26.520131 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21285115-6507-4ab7-a461-b36b673a0cb4 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:26.520314 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21285115-6507-4ab7-a461-b36b673a0cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:26.537750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-21285115-6507-4ab7-a461-b36b673a0cb4 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:26.539281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-21285115-6507-4ab7-a461-b36b673a0cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:26.609504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 474/951] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:26.632259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fd890-44a5-4712-8950-3534e7a98e2d None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:26.737876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fd890-44a5-4712-8950-3534e7a98e2d None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:26.775751 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c5b22-94cf-4b66-bcf5-5341398727cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:26.776469 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c5b22-94cf-4b66-bcf5-5341398727cb None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:26.776750 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c5b22-94cf-4b66-bcf5-5341398727cb None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:26.777027 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c5b22-94cf-4b66-bcf5-5341398727cb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:26.777678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c5b22-94cf-4b66-bcf5-5341398727cb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:26.779654 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c5b22-94cf-4b66-bcf5-5341398727cb None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-528c5b22-94cf-4b66-bcf5-5341398727cb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:26.791670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:26.792673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:26.793492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 478/952] 10.222.0.35 () {56 vars in 1126 bytes} [Fri May 13 21:39:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:26.799289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21285115-6507-4ab7-a461-b36b673a0cb4 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:26.900927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21285115-6507-4ab7-a461-b36b673a0cb4 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:26.934795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f894e6-a6d3-42e7-992d-2f60632eb32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:26.935456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f894e6-a6d3-42e7-992d-2f60632eb32b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:26.935741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f894e6-a6d3-42e7-992d-2f60632eb32b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:26.936019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f894e6-a6d3-42e7-992d-2f60632eb32b None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:26.936721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f894e6-a6d3-42e7-992d-2f60632eb32b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:26.937026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f894e6-a6d3-42e7-992d-2f60632eb32b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:26.938396 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f894e6-a6d3-42e7-992d-2f60632eb32b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:26.943536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 475/953] 10.222.0.35 () {56 vars in 1122 bytes} [Fri May 13 21:39:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:26.948944 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c5b22-94cf-4b66-bcf5-5341398727cb None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:27.051216 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c5b22-94cf-4b66-bcf5-5341398727cb None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:27.085355 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e431fb-854a-49de-91b7-ac140f16fc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:27.086017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e431fb-854a-49de-91b7-ac140f16fc83 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:27.086297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e431fb-854a-49de-91b7-ac140f16fc83 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:27.086573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e431fb-854a-49de-91b7-ac140f16fc83 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:27.087326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e431fb-854a-49de-91b7-ac140f16fc83 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:27.087605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e431fb-854a-49de-91b7-ac140f16fc83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:27.088978 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e431fb-854a-49de-91b7-ac140f16fc83 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:27.094861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 479/954] 10.222.0.35 () {56 vars in 1122 bytes} [Fri May 13 21:39:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:27.100535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f894e6-a6d3-42e7-992d-2f60632eb32b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:27.216399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f894e6-a6d3-42e7-992d-2f60632eb32b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:27.251995 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da00bd2-d835-4631-8899-fc730339d3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:27.252660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da00bd2-d835-4631-8899-fc730339d3c4 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:27.252942 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da00bd2-d835-4631-8899-fc730339d3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:27.253218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da00bd2-d835-4631-8899-fc730339d3c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:27.253969 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da00bd2-d835-4631-8899-fc730339d3c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:27.255379 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da00bd2-d835-4631-8899-fc730339d3c4 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:27.277534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 476/955] 10.222.0.35 () {60 vars in 1144 bytes} [Fri May 13 21:39:27 2022] POST /identity/v3/endpoints => generated 349 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:39:27.365943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a5edc-fdc9-4641-90af-01d706711ac1 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:27.366244 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a5edc-fdc9-4641-90af-01d706711ac1 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:27.366417 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a5edc-fdc9-4641-90af-01d706711ac1 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:27.368105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 480/956] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:27.373782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95851624-a1c7-413f-9ebd-00cf93ebf478 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:27.374550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95851624-a1c7-413f-9ebd-00cf93ebf478 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:27.375035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95851624-a1c7-413f-9ebd-00cf93ebf478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:27.393273 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-95851624-a1c7-413f-9ebd-00cf93ebf478 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:27.394779 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-95851624-a1c7-413f-9ebd-00cf93ebf478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:27.480333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 477/957] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:27.505102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a5edc-fdc9-4641-90af-01d706711ac1 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:27.616947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a5edc-fdc9-4641-90af-01d706711ac1 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:27.651082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d90d4-8f95-4caf-98ed-12dad02f1cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:27.651802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d90d4-8f95-4caf-98ed-12dad02f1cc2 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:27.652084 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d90d4-8f95-4caf-98ed-12dad02f1cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:27.652355 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d90d4-8f95-4caf-98ed-12dad02f1cc2 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:27.653051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d90d4-8f95-4caf-98ed-12dad02f1cc2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:27.655194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d90d4-8f95-4caf-98ed-12dad02f1cc2 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:27.656599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: WARNING keystone.server.flask.common [None req-2f4d90d4-8f95-4caf-98ed-12dad02f1cc2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 13 21:39:27.669775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.sql.core [None req-2f4d90d4-8f95-4caf-98ed-12dad02f1cc2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 13 21:39:27.669775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 13 21:39:27.669775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [parameters: {'user_id': 'ded9be201f1840b3a1a25732d18db3a7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 13 21:39:27.669775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91430) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-2f4d90d4-8f95-4caf-98ed-12dad02f1cc2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application result = self._query(query) May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application conn.query(q) May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application result.read() May 13 21:39:27.678276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 21:39:27.679281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application next(self.gen) May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application yield resource May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application next(self.gen) May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self.session.rollback() May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self.force_reraise() May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise self.value May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application session.commit() May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 13 21:39:27.680253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._prepare_impl() May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self.session.flush() May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._flush(objects) May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application compat.raise_( May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application flush_context.execute() May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rec.execute(self) May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 13 21:39:27.681144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application _emit_insert_statements( May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application result = connection._execute_20( May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ret = self._execute_context( May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 21:39:27.682000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application result = self._query(query) May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application conn.query(q) May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application result.read() May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 21:39:27.683985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ded9be201f1840b3a1a25732d18db3a7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:27.686036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 13 21:39:27.687965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 13 21:39:27.698603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 13 21:39:27.698603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 13 21:39:27.698603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 13 21:39:27.698603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 13 21:39:27.698603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 13 21:39:27.698603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:27.698603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 481/958] 10.222.0.35 () {60 vars in 1131 bytes} [Fri May 13 21:39:27 2022] POST /identity/v3/users => generated 160 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:27.699562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95851624-a1c7-413f-9ebd-00cf93ebf478 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:27.773462 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef90267-0327-4644-a7cc-d2dda98a0be1 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:27.773814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef90267-0327-4644-a7cc-d2dda98a0be1 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:27.774004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef90267-0327-4644-a7cc-d2dda98a0be1 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:27.780366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 482/959] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:27.785707 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dda4c8-7b88-45ef-a914-42e8856dcd82 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:27.786259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dda4c8-7b88-45ef-a914-42e8856dcd82 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:27.786794 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dda4c8-7b88-45ef-a914-42e8856dcd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:27.803635 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95851624-a1c7-413f-9ebd-00cf93ebf478 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:27.809023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-29dda4c8-7b88-45ef-a914-42e8856dcd82 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:27.811828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-29dda4c8-7b88-45ef-a914-42e8856dcd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:27.847325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9f442-4b64-4179-810d-b75c6264540b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:27.848120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9f442-4b64-4179-810d-b75c6264540b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:27.848402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9f442-4b64-4179-810d-b75c6264540b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:27.848683 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9f442-4b64-4179-810d-b75c6264540b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:27.857197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9f442-4b64-4179-810d-b75c6264540b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:27.858985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9f442-4b64-4179-810d-b75c6264540b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-00a9f442-4b64-4179-810d-b75c6264540b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:27.876354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:27.877934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:27.878940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:27.878940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:27.878940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:27.878940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:27.878940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:27.878940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:27.878940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:27.878940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:27.878940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 13 21:39:27.878940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:27.879854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 478/960] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:39:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:27.880723 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247b288-bfae-437a-ab29-323d419286c9 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:27.881009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247b288-bfae-437a-ab29-323d419286c9 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:27.881281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247b288-bfae-437a-ab29-323d419286c9 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:27.882697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 479/961] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:27.884523 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247b288-bfae-437a-ab29-323d419286c9 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:27.894781 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 483/962] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:27.894781 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71872921-ee92-441f-b49c-94bbd6592714 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:27.894781 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71872921-ee92-441f-b49c-94bbd6592714 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:27.894781 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71872921-ee92-441f-b49c-94bbd6592714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:27.913780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-71872921-ee92-441f-b49c-94bbd6592714 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:27.915268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-71872921-ee92-441f-b49c-94bbd6592714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:27.982435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 484/963] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:27.983086 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71872921-ee92-441f-b49c-94bbd6592714 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:27.996468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247b288-bfae-437a-ab29-323d419286c9 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:28.035353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f7170-1edc-4f68-b132-4db502d0233c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:28.036026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f7170-1edc-4f68-b132-4db502d0233c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:28.036308 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f7170-1edc-4f68-b132-4db502d0233c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:28.036586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f7170-1edc-4f68-b132-4db502d0233c None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:28.037391 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f7170-1edc-4f68-b132-4db502d0233c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:28.037695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f7170-1edc-4f68-b132-4db502d0233c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:28.039184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f7170-1edc-4f68-b132-4db502d0233c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:28.078744 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 480/964] 10.222.0.35 () {56 vars in 1101 bytes} [Fri May 13 21:39:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:28.078744 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f7170-1edc-4f68-b132-4db502d0233c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:28.097242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71872921-ee92-441f-b49c-94bbd6592714 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:28.147472 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82163bfb-9553-4df0-a2bd-82b1b75c3350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:28.148202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82163bfb-9553-4df0-a2bd-82b1b75c3350 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:28.148482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82163bfb-9553-4df0-a2bd-82b1b75c3350 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:28.148755 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82163bfb-9553-4df0-a2bd-82b1b75c3350 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:28.150786 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82163bfb-9553-4df0-a2bd-82b1b75c3350 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:28.151061 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82163bfb-9553-4df0-a2bd-82b1b75c3350 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-82163bfb-9553-4df0-a2bd-82b1b75c3350 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:28.170046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:28.171149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:28.171995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:28.171995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:28.171995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:28.171995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:28.171995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:28.171995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:28.171995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:28.171995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:28.171995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:28.171995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:28.171995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:28.171995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:28.171995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 21:39:28.171995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:28.176542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 485/965] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:28.177831 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82163bfb-9553-4df0-a2bd-82b1b75c3350 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:28.212222 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f7170-1edc-4f68-b132-4db502d0233c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:28.259921 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f20e4-6205-4f88-aaaf-81db7476e75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:28.261088 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f20e4-6205-4f88-aaaf-81db7476e75c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:28.261574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f20e4-6205-4f88-aaaf-81db7476e75c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:28.262045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f20e4-6205-4f88-aaaf-81db7476e75c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:28.277040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f20e4-6205-4f88-aaaf-81db7476e75c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-610f20e4-6205-4f88-aaaf-81db7476e75c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:28.286492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:28.287538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:28.288364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:28.288364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:28.288364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:28.288364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:28.288364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:28.288364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:28.288364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:28.288364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:28.288364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:28.288364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:28.288364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 21:39:28.288364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:28.288364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 481/966] 10.222.0.35 () {56 vars in 1120 bytes} [Fri May 13 21:39:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:28.293227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82163bfb-9553-4df0-a2bd-82b1b75c3350 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:28.293720 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f20e4-6205-4f88-aaaf-81db7476e75c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:28.334885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14e801-7b96-487d-83e6-c6d38c365584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:28.335781 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14e801-7b96-487d-83e6-c6d38c365584 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:28.336259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14e801-7b96-487d-83e6-c6d38c365584 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:28.336691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14e801-7b96-487d-83e6-c6d38c365584 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:28.337543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14e801-7b96-487d-83e6-c6d38c365584 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:28.337992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14e801-7b96-487d-83e6-c6d38c365584 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:28.339540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14e801-7b96-487d-83e6-c6d38c365584 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:28.345731 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 486/967] 10.222.0.35 () {56 vars in 1112 bytes} [Fri May 13 21:39:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:28.351464 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14e801-7b96-487d-83e6-c6d38c365584 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:28.409549 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f20e4-6205-4f88-aaaf-81db7476e75c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:28.442944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d71ed-b262-468b-880d-babeb1c430cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:28.443607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d71ed-b262-468b-880d-babeb1c430cf None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:28.443944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d71ed-b262-468b-880d-babeb1c430cf None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:28.444228 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d71ed-b262-468b-880d-babeb1c430cf None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:28.445018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d71ed-b262-468b-880d-babeb1c430cf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:28.445326 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d71ed-b262-468b-880d-babeb1c430cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:28.446734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d71ed-b262-468b-880d-babeb1c430cf None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:28.456088 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 482/968] 10.222.0.35 () {56 vars in 1115 bytes} [Fri May 13 21:39:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:28.461629 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d71ed-b262-468b-880d-babeb1c430cf None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:28.464345 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14e801-7b96-487d-83e6-c6d38c365584 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:28.504627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7df9eb-69b2-49d6-bb66-c8835ac24e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:28.505293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7df9eb-69b2-49d6-bb66-c8835ac24e18 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:28.505565 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7df9eb-69b2-49d6-bb66-c8835ac24e18 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:28.505841 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7df9eb-69b2-49d6-bb66-c8835ac24e18 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:28.506532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7df9eb-69b2-49d6-bb66-c8835ac24e18 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:28.506866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7df9eb-69b2-49d6-bb66-c8835ac24e18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:28.508235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7df9eb-69b2-49d6-bb66-c8835ac24e18 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:28.515096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 487/969] 10.222.0.35 () {56 vars in 1112 bytes} [Fri May 13 21:39:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:28.567726 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d71ed-b262-468b-880d-babeb1c430cf None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:28.602794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081c5a5-859c-4cfd-89d1-6cbfd0846b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:28.603600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081c5a5-859c-4cfd-89d1-6cbfd0846b5b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:28.603883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081c5a5-859c-4cfd-89d1-6cbfd0846b5b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:28.604158 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081c5a5-859c-4cfd-89d1-6cbfd0846b5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:28.615848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081c5a5-859c-4cfd-89d1-6cbfd0846b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:28.617340 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081c5a5-859c-4cfd-89d1-6cbfd0846b5b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-e081c5a5-859c-4cfd-89d1-6cbfd0846b5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:28.636002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:28.637042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:28.643460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:28.643460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:28.643460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:28.643460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:28.643460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 483/970] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:28.645474 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7df9eb-69b2-49d6-bb66-c8835ac24e18 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:28.745172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7df9eb-69b2-49d6-bb66-c8835ac24e18 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:28.776741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82930b-eb04-4ca2-b9a0-d439e8e0e53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:28.777394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82930b-eb04-4ca2-b9a0-d439e8e0e53b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:28.777671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82930b-eb04-4ca2-b9a0-d439e8e0e53b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:28.777942 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82930b-eb04-4ca2-b9a0-d439e8e0e53b None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:28.779217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82930b-eb04-4ca2-b9a0-d439e8e0e53b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:28.779522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82930b-eb04-4ca2-b9a0-d439e8e0e53b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:28.780886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82930b-eb04-4ca2-b9a0-d439e8e0e53b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:28.793362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 488/971] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:28.799220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081c5a5-859c-4cfd-89d1-6cbfd0846b5b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:28.893860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081c5a5-859c-4cfd-89d1-6cbfd0846b5b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:28.926350 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6faa161-ad9b-4e36-b8bd-5451daec6fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:28.927182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6faa161-ad9b-4e36-b8bd-5451daec6fa9 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:28.927461 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6faa161-ad9b-4e36-b8bd-5451daec6fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:28.927731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6faa161-ad9b-4e36-b8bd-5451daec6fa9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:28.935031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6faa161-ad9b-4e36-b8bd-5451daec6fa9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:28.936476 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6faa161-ad9b-4e36-b8bd-5451daec6fa9 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-b6faa161-ad9b-4e36-b8bd-5451daec6fa9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:28.950834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:28.951757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:28.952727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:28.952727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:28.952727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:28.952727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:28.952727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:28.952727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:28.952727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:28.952727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:28.952727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 21:39:28.952727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:28.954822 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 484/972] 10.222.0.35 () {56 vars in 1117 bytes} [Fri May 13 21:39:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:28.957638 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82930b-eb04-4ca2-b9a0-d439e8e0e53b None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:29.052714 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82930b-eb04-4ca2-b9a0-d439e8e0e53b None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:29.084928 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6b533-974e-4985-ae3f-c259a880b3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:29.085590 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6b533-974e-4985-ae3f-c259a880b3bb None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:29.085867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6b533-974e-4985-ae3f-c259a880b3bb None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:29.086136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6b533-974e-4985-ae3f-c259a880b3bb None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:29.086980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6b533-974e-4985-ae3f-c259a880b3bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:29.087285 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6b533-974e-4985-ae3f-c259a880b3bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:29.088708 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6b533-974e-4985-ae3f-c259a880b3bb None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:29.110479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 489/973] 10.222.0.35 () {56 vars in 1149 bytes} [Fri May 13 21:39:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:29.111923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6faa161-ad9b-4e36-b8bd-5451daec6fa9 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:29.211639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6faa161-ad9b-4e36-b8bd-5451daec6fa9 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:29.243458 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be8365-1792-4694-9c7c-b100c7801870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:29.244102 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be8365-1792-4694-9c7c-b100c7801870 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:29.244383 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be8365-1792-4694-9c7c-b100c7801870 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:29.244654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be8365-1792-4694-9c7c-b100c7801870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:29.310159 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be8365-1792-4694-9c7c-b100c7801870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:29.311582 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be8365-1792-4694-9c7c-b100c7801870 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:29.405504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 485/974] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:39:29 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 21:39:29.411219 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6b533-974e-4985-ae3f-c259a880b3bb None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:29.506676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6b533-974e-4985-ae3f-c259a880b3bb None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:29.538404 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94d891-8551-4efc-a234-dc20726fee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:29.539198 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94d891-8551-4efc-a234-dc20726fee13 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:29.539476 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94d891-8551-4efc-a234-dc20726fee13 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:29.539750 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94d891-8551-4efc-a234-dc20726fee13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:29.549994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94d891-8551-4efc-a234-dc20726fee13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:29.551603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94d891-8551-4efc-a234-dc20726fee13 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-4e94d891-8551-4efc-a234-dc20726fee13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:29.569667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:29.570563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:29.571822 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:29.571822 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:29.571822 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:29.571822 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:29.575032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 490/975] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:29.576408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be8365-1792-4694-9c7c-b100c7801870 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:29.589812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec99d1c-7487-4cf7-8240-d13fdb21e4cc None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:29.590177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec99d1c-7487-4cf7-8240-d13fdb21e4cc None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:29.590379 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec99d1c-7487-4cf7-8240-d13fdb21e4cc None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:29.592292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 491/976] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:39:29.597321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7e98b-97ff-410a-b577-e9b2a3264a4b None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:29.597612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7e98b-97ff-410a-b577-e9b2a3264a4b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:29.597887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7e98b-97ff-410a-b577-e9b2a3264a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:29.614617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-08f7e98b-97ff-410a-b577-e9b2a3264a4b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:29.615843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-08f7e98b-97ff-410a-b577-e9b2a3264a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:29.678139 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be8365-1792-4694-9c7c-b100c7801870 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:29.682174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 492/977] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:29.704975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7e98b-97ff-410a-b577-e9b2a3264a4b None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:29.711852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8108f-a0de-4ba0-8d67-d1ee477bc60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:29.712523 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8108f-a0de-4ba0-8d67-d1ee477bc60b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:29.712800 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8108f-a0de-4ba0-8d67-d1ee477bc60b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:29.713073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8108f-a0de-4ba0-8d67-d1ee477bc60b None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:29.713792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8108f-a0de-4ba0-8d67-d1ee477bc60b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:29.714097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8108f-a0de-4ba0-8d67-d1ee477bc60b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:29.715535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8108f-a0de-4ba0-8d67-d1ee477bc60b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:29.727524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 486/978] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:29.732758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8108f-a0de-4ba0-8d67-d1ee477bc60b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:29.804871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7e98b-97ff-410a-b577-e9b2a3264a4b None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:29.828703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8108f-a0de-4ba0-8d67-d1ee477bc60b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:29.842006 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339063f-5f34-4b14-8e59-5f25d3ec393c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:29.842829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339063f-5f34-4b14-8e59-5f25d3ec393c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:29.843110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339063f-5f34-4b14-8e59-5f25d3ec393c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:29.843381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339063f-5f34-4b14-8e59-5f25d3ec393c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:29.844022 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339063f-5f34-4b14-8e59-5f25d3ec393c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:29.845552 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339063f-5f34-4b14-8e59-5f25d3ec393c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-8339063f-5f34-4b14-8e59-5f25d3ec393c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:29.857150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:29.858194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:29.859123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 493/979] 10.222.0.35 () {56 vars in 1126 bytes} [Fri May 13 21:39:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:29.864084 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339063f-5f34-4b14-8e59-5f25d3ec393c None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:29.864568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890639b-184d-4803-9fcf-a30e1fb51532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:29.865359 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890639b-184d-4803-9fcf-a30e1fb51532 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:29.865639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890639b-184d-4803-9fcf-a30e1fb51532 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:29.865911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890639b-184d-4803-9fcf-a30e1fb51532 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:29.872849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890639b-184d-4803-9fcf-a30e1fb51532 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:29.874296 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890639b-184d-4803-9fcf-a30e1fb51532 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-5890639b-184d-4803-9fcf-a30e1fb51532 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:29.889074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:29.889997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:29.891076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:29.891076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:29.891076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:29.891076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:29.891076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:29.891076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:29.891076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:29.891076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:29.891076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 487/980] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:29.895912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890639b-184d-4803-9fcf-a30e1fb51532 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:29.963079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339063f-5f34-4b14-8e59-5f25d3ec393c None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:29.989761 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890639b-184d-4803-9fcf-a30e1fb51532 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:29.996343 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb8336-0671-41e2-ae2c-3a0649b238fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:29.997024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb8336-0671-41e2-ae2c-3a0649b238fe None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:29.997311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb8336-0671-41e2-ae2c-3a0649b238fe None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:29.997590 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb8336-0671-41e2-ae2c-3a0649b238fe None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:29.998286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb8336-0671-41e2-ae2c-3a0649b238fe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:29.998597 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb8336-0671-41e2-ae2c-3a0649b238fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:30.000000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb8336-0671-41e2-ae2c-3a0649b238fe None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:30.009254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 494/981] 10.222.0.35 () {56 vars in 1122 bytes} [Fri May 13 21:39:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:30.010472 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb8336-0671-41e2-ae2c-3a0649b238fe None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:30.024431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4476ea4-ccc1-46c2-875a-736b5cbf5fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:30.025086 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4476ea4-ccc1-46c2-875a-736b5cbf5fe5 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:30.025363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4476ea4-ccc1-46c2-875a-736b5cbf5fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:30.025624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4476ea4-ccc1-46c2-875a-736b5cbf5fe5 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:30.026399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4476ea4-ccc1-46c2-875a-736b5cbf5fe5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:30.026731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4476ea4-ccc1-46c2-875a-736b5cbf5fe5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:30.028726 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4476ea4-ccc1-46c2-875a-736b5cbf5fe5 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:30.044775 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 488/982] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:30.045852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4476ea4-ccc1-46c2-875a-736b5cbf5fe5 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:30.111484 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb8336-0671-41e2-ae2c-3a0649b238fe None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:30.144203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fece35-36c9-4a98-8ec7-e6ddf703a0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:30.144852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fece35-36c9-4a98-8ec7-e6ddf703a0e1 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:30.145130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fece35-36c9-4a98-8ec7-e6ddf703a0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:30.145402 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fece35-36c9-4a98-8ec7-e6ddf703a0e1 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:30.146089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fece35-36c9-4a98-8ec7-e6ddf703a0e1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:30.146391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fece35-36c9-4a98-8ec7-e6ddf703a0e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:30.147790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fece35-36c9-4a98-8ec7-e6ddf703a0e1 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:30.155665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 495/983] 10.222.0.35 () {56 vars in 1122 bytes} [Fri May 13 21:39:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:30.157421 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4476ea4-ccc1-46c2-875a-736b5cbf5fe5 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:30.161097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fece35-36c9-4a98-8ec7-e6ddf703a0e1 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:30.194877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661f716-8d9c-49f4-9936-148b80c40f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:30.195542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661f716-8d9c-49f4-9936-148b80c40f1d None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:30.195832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661f716-8d9c-49f4-9936-148b80c40f1d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:30.196095 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661f716-8d9c-49f4-9936-148b80c40f1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:30.196928 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661f716-8d9c-49f4-9936-148b80c40f1d None admin] RBAC: Adding query filter params (role.id=708710710e3e44dca3a6d11cbc2b6dd6, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=a079ae1557d04831b73c03c96e2c7ad6) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:30.197243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661f716-8d9c-49f4-9936-148b80c40f1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:30.199313 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661f716-8d9c-49f4-9936-148b80c40f1d None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:30.214745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 489/984] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:39:30 2022] GET /identity/v3/role_assignments?user.id=a079ae1557d04831b73c03c96e2c7ad6&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=708710710e3e44dca3a6d11cbc2b6dd6 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:30.265558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fece35-36c9-4a98-8ec7-e6ddf703a0e1 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:30.300663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d420d-edbc-4211-a587-8f3ca0342cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:30.301333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d420d-edbc-4211-a587-8f3ca0342cb1 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:30.301610 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d420d-edbc-4211-a587-8f3ca0342cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:30.301880 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d420d-edbc-4211-a587-8f3ca0342cb1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:30.302676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d420d-edbc-4211-a587-8f3ca0342cb1 None admin] RBAC: Adding query filter params (interface=admin, service_id=a46e47859a1441daa0c4dc7279aa1f4f, region_id=RegionOne) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:30.303017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d420d-edbc-4211-a587-8f3ca0342cb1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:30.304527 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d420d-edbc-4211-a587-8f3ca0342cb1 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:30.312059 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 496/985] 10.222.0.35 () {56 vars in 1249 bytes} [Fri May 13 21:39:30 2022] GET /identity/v3/endpoints?service_id=a46e47859a1441daa0c4dc7279aa1f4f&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:30.317548 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661f716-8d9c-49f4-9936-148b80c40f1d None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:30.418542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661f716-8d9c-49f4-9936-148b80c40f1d None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:30.453606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925d8a0-f165-4f09-9503-dfd6d8ba35ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:30.454272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925d8a0-f165-4f09-9503-dfd6d8ba35ac None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:30.454557 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925d8a0-f165-4f09-9503-dfd6d8ba35ac None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:30.454911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925d8a0-f165-4f09-9503-dfd6d8ba35ac None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:30.455598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925d8a0-f165-4f09-9503-dfd6d8ba35ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:30.456985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925d8a0-f165-4f09-9503-dfd6d8ba35ac None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:30.463062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 490/986] 10.222.0.35 () {56 vars in 1087 bytes} [Fri May 13 21:39:30 2022] GET /identity/v3/services => generated 1497 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:39:30.742318 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19065c19-c487-45cc-ad83-e9239f77da55 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:30.742888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19065c19-c487-45cc-ad83-e9239f77da55 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:30.742888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19065c19-c487-45cc-ad83-e9239f77da55 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:30.744398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 497/987] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:30.750150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc5b79-effb-43de-ba79-733b9171df28 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:30.750443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc5b79-effb-43de-ba79-733b9171df28 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:30.750734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc5b79-effb-43de-ba79-733b9171df28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:30.772073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-65bc5b79-effb-43de-ba79-733b9171df28 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:30.773263 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-65bc5b79-effb-43de-ba79-733b9171df28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:30.837458 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 491/988] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:30.860892 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19065c19-c487-45cc-ad83-e9239f77da55 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:30.964192 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19065c19-c487-45cc-ad83-e9239f77da55 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:30.997207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269f2cd-82e2-4d1b-8572-45a3b83296e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:30.997784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269f2cd-82e2-4d1b-8572-45a3b83296e6 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:30.998063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269f2cd-82e2-4d1b-8572-45a3b83296e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:30.998335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269f2cd-82e2-4d1b-8572-45a3b83296e6 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:30.999075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269f2cd-82e2-4d1b-8572-45a3b83296e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:31.000436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269f2cd-82e2-4d1b-8572-45a3b83296e6 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:31.016160 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 498/989] 10.222.0.35 () {60 vars in 1141 bytes} [Fri May 13 21:39:30 2022] POST /identity/v3/services => generated 242 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:39:32.443662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604764eb-31cf-410f-8e96-d9962a535cd4 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:32.444163 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604764eb-31cf-410f-8e96-d9962a535cd4 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:32.444240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604764eb-31cf-410f-8e96-d9962a535cd4 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:32.446760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 492/990] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:32.451594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e276f-ca77-4653-a9d5-bfd1131304fd None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:32.452015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e276f-ca77-4653-a9d5-bfd1131304fd None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:32.452178 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e276f-ca77-4653-a9d5-bfd1131304fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:32.468222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d30e276f-ca77-4653-a9d5-bfd1131304fd None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:32.469375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d30e276f-ca77-4653-a9d5-bfd1131304fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:32.543325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 499/991] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:32.565507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604764eb-31cf-410f-8e96-d9962a535cd4 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:32.661396 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604764eb-31cf-410f-8e96-d9962a535cd4 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:32.690992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3f503-af1b-4dcf-b2b8-2abc1a3441ca None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:32.691339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3f503-af1b-4dcf-b2b8-2abc1a3441ca None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:32.691565 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3f503-af1b-4dcf-b2b8-2abc1a3441ca None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:32.693111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 500/992] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:39:32.693761 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbd391-bb0b-426e-bfed-ba72ec932b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:32.694551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbd391-bb0b-426e-bfed-ba72ec932b90 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:32.694863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbd391-bb0b-426e-bfed-ba72ec932b90 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:32.695125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbd391-bb0b-426e-bfed-ba72ec932b90 None admin] PATH_INFO: `/v3/services/image` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:32.695768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbd391-bb0b-426e-bfed-ba72ec932b90 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:32.697227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbd391-bb0b-426e-bfed-ba72ec932b90 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:32.697706 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008bd1a-cbd7-41e1-91ca-e82234cc973d None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:32.697996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008bd1a-cbd7-41e1-91ca-e82234cc973d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:32.698270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008bd1a-cbd7-41e1-91ca-e82234cc973d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-2bfbd391-bb0b-426e-bfed-ba72ec932b90 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:32.709252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:32.710453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:32.711551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:32.711551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:32.711551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:32.711551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:32.711551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:32.711551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:32.711551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:32.711551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:32.711551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:32.711551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:32.711551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:32.711551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:32.711551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 21:39:32.711551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:32.714882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 493/993] 10.222.0.35 () {56 vars in 1105 bytes} [Fri May 13 21:39:32 2022] GET /identity/v3/services/image => generated 86 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:32.716451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbd391-bb0b-426e-bfed-ba72ec932b90 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:32.718298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e008bd1a-cbd7-41e1-91ca-e82234cc973d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:32.719509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e008bd1a-cbd7-41e1-91ca-e82234cc973d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:32.798745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 501/994] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:32.818324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008bd1a-cbd7-41e1-91ca-e82234cc973d None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:32.827534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbd391-bb0b-426e-bfed-ba72ec932b90 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:32.860267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a8814-181e-470b-b532-d88786536d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:32.860927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a8814-181e-470b-b532-d88786536d60 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:32.861212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a8814-181e-470b-b532-d88786536d60 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:32.861491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a8814-181e-470b-b532-d88786536d60 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:32.862195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a8814-181e-470b-b532-d88786536d60 None admin] RBAC: Adding query filter params (name=image) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:32.862532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a8814-181e-470b-b532-d88786536d60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:32.863924 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a8814-181e-470b-b532-d88786536d60 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:32.873598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 494/995] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:32.874834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a8814-181e-470b-b532-d88786536d60 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:32.911922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008bd1a-cbd7-41e1-91ca-e82234cc973d None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:32.945116 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dbea4-71ed-4643-8b97-ffbb25197491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:32.945834 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dbea4-71ed-4643-8b97-ffbb25197491 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:32.946111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dbea4-71ed-4643-8b97-ffbb25197491 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:32.946384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dbea4-71ed-4643-8b97-ffbb25197491 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:32.947053 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079dbea4-71ed-4643-8b97-ffbb25197491 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:32.948423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079dbea4-71ed-4643-8b97-ffbb25197491 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-079dbea4-71ed-4643-8b97-ffbb25197491 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:32.960038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:32.961025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:32.967409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 502/996] 10.222.0.35 () {56 vars in 1126 bytes} [Fri May 13 21:39:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:32.968189 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dbea4-71ed-4643-8b97-ffbb25197491 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:32.970432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a8814-181e-470b-b532-d88786536d60 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:33.003274 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a40b3-1d7f-44bc-84fb-572145471f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:33.003931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a40b3-1d7f-44bc-84fb-572145471f5c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:33.004214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a40b3-1d7f-44bc-84fb-572145471f5c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:33.004490 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a40b3-1d7f-44bc-84fb-572145471f5c None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:33.005186 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a40b3-1d7f-44bc-84fb-572145471f5c None admin] RBAC: Adding query filter params (type=image) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:33.005492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a40b3-1d7f-44bc-84fb-572145471f5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:33.006883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a40b3-1d7f-44bc-84fb-572145471f5c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:33.011170 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 495/997] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:32 2022] GET /identity/v3/services?type=image => generated 122 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:33.078331 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dbea4-71ed-4643-8b97-ffbb25197491 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:33.115352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46206788-0ef8-47fa-ae4f-7b72abc4519a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:33.116027 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46206788-0ef8-47fa-ae4f-7b72abc4519a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:33.116309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46206788-0ef8-47fa-ae4f-7b72abc4519a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:33.116584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46206788-0ef8-47fa-ae4f-7b72abc4519a None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:33.117312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46206788-0ef8-47fa-ae4f-7b72abc4519a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:33.117617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46206788-0ef8-47fa-ae4f-7b72abc4519a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:33.119081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46206788-0ef8-47fa-ae4f-7b72abc4519a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:33.129422 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 503/998] 10.222.0.35 () {56 vars in 1122 bytes} [Fri May 13 21:39:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:33.131206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a40b3-1d7f-44bc-84fb-572145471f5c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:33.244402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a40b3-1d7f-44bc-84fb-572145471f5c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:33.261627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023eb52-1b93-456c-a16d-ef83e7d5640f None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:33.261918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023eb52-1b93-456c-a16d-ef83e7d5640f None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:33.262195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023eb52-1b93-456c-a16d-ef83e7d5640f None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:33.263782 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 504/999] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:39:33.268474 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f951c7-5030-488f-9d38-befbaf55f994 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:33.268763 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f951c7-5030-488f-9d38-befbaf55f994 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:33.269043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f951c7-5030-488f-9d38-befbaf55f994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:33.276284 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d472f-930d-49a6-9215-8b7440212b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:33.276940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d472f-930d-49a6-9215-8b7440212b5d None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:33.277218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d472f-930d-49a6-9215-8b7440212b5d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:33.277489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d472f-930d-49a6-9215-8b7440212b5d None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:33.278751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d472f-930d-49a6-9215-8b7440212b5d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:33.279056 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d472f-930d-49a6-9215-8b7440212b5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:33.280423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d472f-930d-49a6-9215-8b7440212b5d None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:33.285110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 496/1000] 10.222.0.35 () {56 vars in 1122 bytes} [Fri May 13 21:39:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:33.285984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-77f951c7-5030-488f-9d38-befbaf55f994 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:33.287398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-77f951c7-5030-488f-9d38-befbaf55f994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:33.290512 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d472f-930d-49a6-9215-8b7440212b5d None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:33.351973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 505/1001] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:33.375182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f951c7-5030-488f-9d38-befbaf55f994 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:33.390456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d472f-930d-49a6-9215-8b7440212b5d None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:33.423229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611638a-524d-430c-a8d6-059e01070aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:33.423883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611638a-524d-430c-a8d6-059e01070aa5 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:33.424162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611638a-524d-430c-a8d6-059e01070aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:33.424434 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611638a-524d-430c-a8d6-059e01070aa5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:33.425158 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611638a-524d-430c-a8d6-059e01070aa5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:33.426527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611638a-524d-430c-a8d6-059e01070aa5 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:33.442175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 497/1002] 10.222.0.35 () {60 vars in 1144 bytes} [Fri May 13 21:39:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:39:33.474428 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f951c7-5030-488f-9d38-befbaf55f994 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:33.517763 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b5891-d6ed-4667-8997-00cfc51a7611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:33.518489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b5891-d6ed-4667-8997-00cfc51a7611 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:33.518797 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b5891-d6ed-4667-8997-00cfc51a7611 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:33.519073 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b5891-d6ed-4667-8997-00cfc51a7611 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:33.519730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b5891-d6ed-4667-8997-00cfc51a7611 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:33.521132 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b5891-d6ed-4667-8997-00cfc51a7611 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-6b9b5891-d6ed-4667-8997-00cfc51a7611 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:33.534075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:33.535180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:33.536019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 506/1003] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:33.541539 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611638a-524d-430c-a8d6-059e01070aa5 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:33.647502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611638a-524d-430c-a8d6-059e01070aa5 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:33.681120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d8f1f-4239-4724-9fcc-db02c6a1c500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:33.681787 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d8f1f-4239-4724-9fcc-db02c6a1c500 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:33.682074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d8f1f-4239-4724-9fcc-db02c6a1c500 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:33.682353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d8f1f-4239-4724-9fcc-db02c6a1c500 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:33.683145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d8f1f-4239-4724-9fcc-db02c6a1c500 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:33.683456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d8f1f-4239-4724-9fcc-db02c6a1c500 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:33.684860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d8f1f-4239-4724-9fcc-db02c6a1c500 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:33.689669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 498/1004] 10.222.0.35 () {56 vars in 1112 bytes} [Fri May 13 21:39:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:33.695477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b5891-d6ed-4667-8997-00cfc51a7611 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:33.792215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b5891-d6ed-4667-8997-00cfc51a7611 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:33.824172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7fd2e-7f89-48eb-bc98-ea1f2d4baaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:33.825374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7fd2e-7f89-48eb-bc98-ea1f2d4baaa5 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:33.826828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7fd2e-7f89-48eb-bc98-ea1f2d4baaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:33.826828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7fd2e-7f89-48eb-bc98-ea1f2d4baaa5 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:33.826828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7fd2e-7f89-48eb-bc98-ea1f2d4baaa5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:33.826828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7fd2e-7f89-48eb-bc98-ea1f2d4baaa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:33.828326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7fd2e-7f89-48eb-bc98-ea1f2d4baaa5 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:33.838023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 507/1005] 10.222.0.35 () {56 vars in 1112 bytes} [Fri May 13 21:39:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:33.839266 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d8f1f-4239-4724-9fcc-db02c6a1c500 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:33.935212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d8f1f-4239-4724-9fcc-db02c6a1c500 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:33.967635 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f9206-ca63-4b4c-9169-5e68c2d0a5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:33.968288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f9206-ca63-4b4c-9169-5e68c2d0a5be None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:33.968576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f9206-ca63-4b4c-9169-5e68c2d0a5be None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:33.968855 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f9206-ca63-4b4c-9169-5e68c2d0a5be None admin] PATH_INFO: `/v3/endpoints` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:33.969641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f9206-ca63-4b4c-9169-5e68c2d0a5be None admin] RBAC: Adding query filter params (interface=public, service_id=6f866aca006940d4bcfdef6c4da26457, region_id=RegionOne) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:33.971623 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f9206-ca63-4b4c-9169-5e68c2d0a5be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:33.971623 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f9206-ca63-4b4c-9169-5e68c2d0a5be None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:33.976542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 499/1006] 10.222.0.35 () {56 vars in 1251 bytes} [Fri May 13 21:39:33 2022] GET /identity/v3/endpoints?service_id=6f866aca006940d4bcfdef6c4da26457&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:33.982795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7fd2e-7f89-48eb-bc98-ea1f2d4baaa5 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:34.076575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7fd2e-7f89-48eb-bc98-ea1f2d4baaa5 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:34.108563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9d090-7074-4600-99a2-3a5d713838a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:34.109214 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9d090-7074-4600-99a2-3a5d713838a6 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:34.109496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9d090-7074-4600-99a2-3a5d713838a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:34.109773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9d090-7074-4600-99a2-3a5d713838a6 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:34.110446 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9d090-7074-4600-99a2-3a5d713838a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:34.111840 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9d090-7074-4600-99a2-3a5d713838a6 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:34.122797 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 508/1007] 10.222.0.35 () {56 vars in 1087 bytes} [Fri May 13 21:39:33 2022] GET /identity/v3/services => generated 1727 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:39:35.278595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b3fa5-f8a7-49d0-bb54-df8df56a75fd None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:35.279169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b3fa5-f8a7-49d0-bb54-df8df56a75fd None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:35.279169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b3fa5-f8a7-49d0-bb54-df8df56a75fd None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:35.280787 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 500/1008] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:35.286424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475ef09-fdee-4e39-8efa-f52fd47506ab None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:35.286942 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475ef09-fdee-4e39-8efa-f52fd47506ab None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:35.287020 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475ef09-fdee-4e39-8efa-f52fd47506ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:35.304022 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-3475ef09-fdee-4e39-8efa-f52fd47506ab None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:35.305200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3475ef09-fdee-4e39-8efa-f52fd47506ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:35.371465 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 509/1009] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:35 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:35.393985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b3fa5-f8a7-49d0-bb54-df8df56a75fd None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:35.492568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b3fa5-f8a7-49d0-bb54-df8df56a75fd None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:35.529222 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a41da4-e4c5-42b9-b55b-527fb9015d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:35.529884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a41da4-e4c5-42b9-b55b-527fb9015d8b None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:35.530168 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a41da4-e4c5-42b9-b55b-527fb9015d8b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:35.530442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a41da4-e4c5-42b9-b55b-527fb9015d8b None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:35.531207 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a41da4-e4c5-42b9-b55b-527fb9015d8b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:35.532572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a41da4-e4c5-42b9-b55b-527fb9015d8b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:35.546933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 501/1010] 10.222.0.35 () {60 vars in 1141 bytes} [Fri May 13 21:39:35 2022] POST /identity/v3/services => generated 242 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:39:35.687919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642e710-aa3c-438d-ae4d-cb3e61da7c5a None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:35.688226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642e710-aa3c-438d-ae4d-cb3e61da7c5a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:35.688657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642e710-aa3c-438d-ae4d-cb3e61da7c5a None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:35.690215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 510/1011] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:35.695998 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfccd0d-fe57-4649-a682-2e4042abf11e None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:35.696280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfccd0d-fe57-4649-a682-2e4042abf11e None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:35.696560 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfccd0d-fe57-4649-a682-2e4042abf11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:35.716555 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-1bfccd0d-fe57-4649-a682-2e4042abf11e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:35.717758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-1bfccd0d-fe57-4649-a682-2e4042abf11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:35.790168 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 502/1012] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:35 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:35.814281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642e710-aa3c-438d-ae4d-cb3e61da7c5a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:35.911739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642e710-aa3c-438d-ae4d-cb3e61da7c5a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:35.943754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee078e7-dec7-4346-9a45-1b398e1ea422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:35.944539 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee078e7-dec7-4346-9a45-1b398e1ea422 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:35.944817 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee078e7-dec7-4346-9a45-1b398e1ea422 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:35.945088 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee078e7-dec7-4346-9a45-1b398e1ea422 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:35.946250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee078e7-dec7-4346-9a45-1b398e1ea422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:35.947843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee078e7-dec7-4346-9a45-1b398e1ea422 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:35.954909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 511/1013] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:35.955940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfccd0d-fe57-4649-a682-2e4042abf11e None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:36.051757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfccd0d-fe57-4649-a682-2e4042abf11e None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:36.086017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe37ff8-8ffb-4ddb-8ded-ff68957b92ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:36.086678 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe37ff8-8ffb-4ddb-8ded-ff68957b92ea None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:36.086997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe37ff8-8ffb-4ddb-8ded-ff68957b92ea None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:36.087275 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe37ff8-8ffb-4ddb-8ded-ff68957b92ea None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:36.087977 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe37ff8-8ffb-4ddb-8ded-ff68957b92ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:36.089407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe37ff8-8ffb-4ddb-8ded-ff68957b92ea None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:36.118749 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 503/1014] 10.222.0.35 () {60 vars in 1141 bytes} [Fri May 13 21:39:35 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:39:36.404997 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e636dbb-a5d0-403c-b0a4-1953a5b887e3 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:36.405501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e636dbb-a5d0-403c-b0a4-1953a5b887e3 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:36.405501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e636dbb-a5d0-403c-b0a4-1953a5b887e3 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:36.406900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 512/1015] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:36.412118 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31197467-2b5d-402c-9d9f-498ce6afc51f None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:36.412497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31197467-2b5d-402c-9d9f-498ce6afc51f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:36.412669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31197467-2b5d-402c-9d9f-498ce6afc51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:36.428983 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-31197467-2b5d-402c-9d9f-498ce6afc51f None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:36.430183 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-31197467-2b5d-402c-9d9f-498ce6afc51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:36.493677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 504/1016] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:36 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:36.516610 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e636dbb-a5d0-403c-b0a4-1953a5b887e3 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:36.614797 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e636dbb-a5d0-403c-b0a4-1953a5b887e3 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:36.647668 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d6858-c3f4-4e2b-96bc-420cfa72d48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:36.648447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6858-c3f4-4e2b-96bc-420cfa72d48e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:36.648721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6858-c3f4-4e2b-96bc-420cfa72d48e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:36.648986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6858-c3f4-4e2b-96bc-420cfa72d48e None admin] PATH_INFO: `/v3/services/compute` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:36.649620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d6858-c3f4-4e2b-96bc-420cfa72d48e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:36.650993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d6858-c3f4-4e2b-96bc-420cfa72d48e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-b58d6858-c3f4-4e2b-96bc-420cfa72d48e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:36.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:36.665032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:36.665873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 513/1017] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:36.670162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31197467-2b5d-402c-9d9f-498ce6afc51f None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:36.771073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31197467-2b5d-402c-9d9f-498ce6afc51f None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:36.804892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dae1c7-7013-4c02-ad5a-6d1bed0a8c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:36.805549 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dae1c7-7013-4c02-ad5a-6d1bed0a8c17 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:36.805821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dae1c7-7013-4c02-ad5a-6d1bed0a8c17 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:36.806096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dae1c7-7013-4c02-ad5a-6d1bed0a8c17 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:36.807112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dae1c7-7013-4c02-ad5a-6d1bed0a8c17 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:36.807417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dae1c7-7013-4c02-ad5a-6d1bed0a8c17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:36.808796 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dae1c7-7013-4c02-ad5a-6d1bed0a8c17 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:36.814432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 505/1018] 10.222.0.35 () {56 vars in 1112 bytes} [Fri May 13 21:39:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:36.819807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d6858-c3f4-4e2b-96bc-420cfa72d48e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:36.915029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d6858-c3f4-4e2b-96bc-420cfa72d48e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:36.947667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a9fa1-d0e7-4d4b-8644-7eb531d788b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:36.948329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a9fa1-d0e7-4d4b-8644-7eb531d788b5 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:36.948608 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a9fa1-d0e7-4d4b-8644-7eb531d788b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:36.948887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a9fa1-d0e7-4d4b-8644-7eb531d788b5 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:36.949576 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a9fa1-d0e7-4d4b-8644-7eb531d788b5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:36.949907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a9fa1-d0e7-4d4b-8644-7eb531d788b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:36.951309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a9fa1-d0e7-4d4b-8644-7eb531d788b5 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:36.956406 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 514/1019] 10.222.0.35 () {56 vars in 1112 bytes} [Fri May 13 21:39:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:36.961731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dae1c7-7013-4c02-ad5a-6d1bed0a8c17 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:37.059734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dae1c7-7013-4c02-ad5a-6d1bed0a8c17 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:37.091619 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6243f-2a5f-4e77-9e54-6ced2619ab86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:37.092338 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6243f-2a5f-4e77-9e54-6ced2619ab86 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:37.092616 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6243f-2a5f-4e77-9e54-6ced2619ab86 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:37.092888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6243f-2a5f-4e77-9e54-6ced2619ab86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:37.093620 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6243f-2a5f-4e77-9e54-6ced2619ab86 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:37.095067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6243f-2a5f-4e77-9e54-6ced2619ab86 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:37.118732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 506/1020] 10.222.0.35 () {60 vars in 1144 bytes} [Fri May 13 21:39:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:39:37.827423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4586e7-a470-4e60-864c-1729bf15e943 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:37.827947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4586e7-a470-4e60-864c-1729bf15e943 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:37.828052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4586e7-a470-4e60-864c-1729bf15e943 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:37.829928 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 515/1021] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:37.836212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b556c90-7513-4024-bd27-17df58e7d066 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:37.836624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b556c90-7513-4024-bd27-17df58e7d066 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:37.836785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b556c90-7513-4024-bd27-17df58e7d066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:37.854057 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-1b556c90-7513-4024-bd27-17df58e7d066 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:37.855270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-1b556c90-7513-4024-bd27-17df58e7d066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:37.926791 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 507/1022] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:37.949065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4586e7-a470-4e60-864c-1729bf15e943 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:38.048027 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4586e7-a470-4e60-864c-1729bf15e943 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:38.079694 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac21f7-649f-4043-93e2-13fa6208c169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:38.080390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac21f7-649f-4043-93e2-13fa6208c169 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:38.080668 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac21f7-649f-4043-93e2-13fa6208c169 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:38.080939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac21f7-649f-4043-93e2-13fa6208c169 None admin] PATH_INFO: `/v3/services/image` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:38.081694 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac21f7-649f-4043-93e2-13fa6208c169 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:38.083081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac21f7-649f-4043-93e2-13fa6208c169 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-8aac21f7-649f-4043-93e2-13fa6208c169 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:38.094838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:38.095812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:38.101526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 516/1023] 10.222.0.35 () {56 vars in 1105 bytes} [Fri May 13 21:39:37 2022] GET /identity/v3/services/image => generated 86 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:38.102305 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b556c90-7513-4024-bd27-17df58e7d066 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:38.203544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b556c90-7513-4024-bd27-17df58e7d066 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:38.238998 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812ad76-c65c-4b63-9251-219f924cfcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:38.239665 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812ad76-c65c-4b63-9251-219f924cfcbb None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:38.239944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812ad76-c65c-4b63-9251-219f924cfcbb None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:38.240216 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812ad76-c65c-4b63-9251-219f924cfcbb None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:38.240916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812ad76-c65c-4b63-9251-219f924cfcbb None admin] RBAC: Adding query filter params (name=image) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:38.241217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812ad76-c65c-4b63-9251-219f924cfcbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:38.242581 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812ad76-c65c-4b63-9251-219f924cfcbb None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:38.252328 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 508/1024] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:38.253235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac21f7-649f-4043-93e2-13fa6208c169 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:38.347439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac21f7-649f-4043-93e2-13fa6208c169 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:38.381758 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a889a7-3497-4dd4-bc35-61aef001907a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:38.382418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a889a7-3497-4dd4-bc35-61aef001907a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:38.382697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a889a7-3497-4dd4-bc35-61aef001907a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:38.382993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a889a7-3497-4dd4-bc35-61aef001907a None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:38.383686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a889a7-3497-4dd4-bc35-61aef001907a None admin] RBAC: Adding query filter params (type=image) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:38.383985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a889a7-3497-4dd4-bc35-61aef001907a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:38.385458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a889a7-3497-4dd4-bc35-61aef001907a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:38.389682 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fa13f-6735-4007-918f-0e710170c684 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:38.389970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fa13f-6735-4007-918f-0e710170c684 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:38.390243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fa13f-6735-4007-918f-0e710170c684 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:38.396031 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 517/1025] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:38.396154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 509/1026] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:39:38.396651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fa13f-6735-4007-918f-0e710170c684 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:38.397106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0fc02-f35a-4975-b856-3449e5dd5101 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:38.397445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0fc02-f35a-4975-b856-3449e5dd5101 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:38.397670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0fc02-f35a-4975-b856-3449e5dd5101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:38.413765 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-15d0fc02-f35a-4975-b856-3449e5dd5101 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:38.414982 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-15d0fc02-f35a-4975-b856-3449e5dd5101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:38.481082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 518/1027] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:38.498379 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fa13f-6735-4007-918f-0e710170c684 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:38.505800 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0fc02-f35a-4975-b856-3449e5dd5101 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:38.535199 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e223436-decd-4228-83fa-eef334d80952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:38.535933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e223436-decd-4228-83fa-eef334d80952 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:38.536222 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e223436-decd-4228-83fa-eef334d80952 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:38.536498 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e223436-decd-4228-83fa-eef334d80952 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:38.537305 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e223436-decd-4228-83fa-eef334d80952 None admin] RBAC: Adding query filter params (interface=public, service_id=0b91468f7f5f4730aa60de9e1c607ec9, region_id=RegionOne) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:38.537617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e223436-decd-4228-83fa-eef334d80952 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:38.539006 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e223436-decd-4228-83fa-eef334d80952 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:38.548420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 510/1028] 10.222.0.35 () {56 vars in 1251 bytes} [Fri May 13 21:39:38 2022] GET /identity/v3/endpoints?service_id=0b91468f7f5f4730aa60de9e1c607ec9&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:38.549851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e223436-decd-4228-83fa-eef334d80952 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:38.603797 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0fc02-f35a-4975-b856-3449e5dd5101 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:38.638895 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d105781-366c-49a6-bca8-a3c432bc56ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:38.639679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d105781-366c-49a6-bca8-a3c432bc56ad None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:38.639966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d105781-366c-49a6-bca8-a3c432bc56ad None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:38.640241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d105781-366c-49a6-bca8-a3c432bc56ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:38.641406 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d105781-366c-49a6-bca8-a3c432bc56ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:38.642876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d105781-366c-49a6-bca8-a3c432bc56ad None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:38.644499 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 519/1029] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:38.646116 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e223436-decd-4228-83fa-eef334d80952 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:38.650040 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d105781-366c-49a6-bca8-a3c432bc56ad None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:38.678577 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681fda0-effe-4ff5-8433-1ec6a2a8d6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:38.679369 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681fda0-effe-4ff5-8433-1ec6a2a8d6cd None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:38.679566 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681fda0-effe-4ff5-8433-1ec6a2a8d6cd None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:38.679933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681fda0-effe-4ff5-8433-1ec6a2a8d6cd None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:38.680513 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681fda0-effe-4ff5-8433-1ec6a2a8d6cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:38.681874 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681fda0-effe-4ff5-8433-1ec6a2a8d6cd None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:38.689406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 511/1030] 10.222.0.35 () {56 vars in 1087 bytes} [Fri May 13 21:39:38 2022] GET /identity/v3/services => generated 1957 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:39:38.749401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d105781-366c-49a6-bca8-a3c432bc56ad None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:38.784669 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd3606-10fb-4ab3-998d-1e630ec80a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:38.785348 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd3606-10fb-4ab3-998d-1e630ec80a44 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:38.785632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd3606-10fb-4ab3-998d-1e630ec80a44 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:38.785906 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd3606-10fb-4ab3-998d-1e630ec80a44 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:38.786627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd3606-10fb-4ab3-998d-1e630ec80a44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:38.788158 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd3606-10fb-4ab3-998d-1e630ec80a44 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:38.811307 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 520/1031] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:39:38 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:39:39.463650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece95ff-7f22-456a-b0ce-bdaa465518b3 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:39.464185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece95ff-7f22-456a-b0ce-bdaa465518b3 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:39.464185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece95ff-7f22-456a-b0ce-bdaa465518b3 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:39.466186 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 512/1032] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:39.471223 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a5661-e746-446b-a6b5-b1ef01d239a0 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:39.471598 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a5661-e746-446b-a6b5-b1ef01d239a0 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:39.471776 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a5661-e746-446b-a6b5-b1ef01d239a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:39.487939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-826a5661-e746-446b-a6b5-b1ef01d239a0 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:39.489102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-826a5661-e746-446b-a6b5-b1ef01d239a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:39.554870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 521/1033] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:39.575569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece95ff-7f22-456a-b0ce-bdaa465518b3 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:39.672934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece95ff-7f22-456a-b0ce-bdaa465518b3 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:39.705268 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14bc67-b6ee-4d29-89fd-b30294745cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:39.706050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14bc67-b6ee-4d29-89fd-b30294745cac None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:39.706333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14bc67-b6ee-4d29-89fd-b30294745cac None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:39.706601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14bc67-b6ee-4d29-89fd-b30294745cac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:39.716996 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14bc67-b6ee-4d29-89fd-b30294745cac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-ef14bc67-b6ee-4d29-89fd-b30294745cac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:39.724981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:39.725973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:39.727203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:39.727203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:39.727203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:39.727203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:39.727203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:39.727203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:39.727203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:39.727203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:39.727203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:39.727203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:39.727203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 21:39:39.727203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:39.731011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 513/1034] 10.222.0.35 () {56 vars in 1120 bytes} [Fri May 13 21:39:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:39.732188 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a5661-e746-446b-a6b5-b1ef01d239a0 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:39.824326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a5661-e746-446b-a6b5-b1ef01d239a0 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:39.866407 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5221157-9327-4688-9c64-0a93edf984b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:39.866407 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5221157-9327-4688-9c64-0a93edf984b8 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:39.866948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5221157-9327-4688-9c64-0a93edf984b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:39.867225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5221157-9327-4688-9c64-0a93edf984b8 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:39.867960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5221157-9327-4688-9c64-0a93edf984b8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:39.868268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5221157-9327-4688-9c64-0a93edf984b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:39.869628 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5221157-9327-4688-9c64-0a93edf984b8 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:39.882413 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 522/1035] 10.222.0.35 () {56 vars in 1115 bytes} [Fri May 13 21:39:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:39.883442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14bc67-b6ee-4d29-89fd-b30294745cac None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:39.977972 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14bc67-b6ee-4d29-89fd-b30294745cac None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:40.012630 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2d879-9651-429a-9046-8783c1c4763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:40.013416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2d879-9651-429a-9046-8783c1c4763e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:40.013695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2d879-9651-429a-9046-8783c1c4763e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:40.013966 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2d879-9651-429a-9046-8783c1c4763e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:40.024195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2d879-9651-429a-9046-8783c1c4763e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:40.025646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2d879-9651-429a-9046-8783c1c4763e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-c5a2d879-9651-429a-9046-8783c1c4763e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:40.051859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:40.052804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:40.059974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:40.059974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:40.059974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:40.059974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:40.059974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 514/1036] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:40.060333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5221157-9327-4688-9c64-0a93edf984b8 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:40.155387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5221157-9327-4688-9c64-0a93edf984b8 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:40.186514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f361c39-6a6f-4c3b-bf89-af4f430458e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:40.187208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f361c39-6a6f-4c3b-bf89-af4f430458e9 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:40.190332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f361c39-6a6f-4c3b-bf89-af4f430458e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:40.190332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f361c39-6a6f-4c3b-bf89-af4f430458e9 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:40.190332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f361c39-6a6f-4c3b-bf89-af4f430458e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:40.190332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f361c39-6a6f-4c3b-bf89-af4f430458e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:40.190332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f361c39-6a6f-4c3b-bf89-af4f430458e9 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:40.206586 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 523/1037] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:40.207784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2d879-9651-429a-9046-8783c1c4763e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:40.302813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2d879-9651-429a-9046-8783c1c4763e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:40.334129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363de90-a6da-49e7-9581-f21f0a27b3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:40.334942 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363de90-a6da-49e7-9581-f21f0a27b3aa None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:40.335227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363de90-a6da-49e7-9581-f21f0a27b3aa None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:40.335573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363de90-a6da-49e7-9581-f21f0a27b3aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:40.342863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363de90-a6da-49e7-9581-f21f0a27b3aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:40.344300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363de90-a6da-49e7-9581-f21f0a27b3aa None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-2363de90-a6da-49e7-9581-f21f0a27b3aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:40.358230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:40.359217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:40.360175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:40.360175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:40.360175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:40.360175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:40.360175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:40.360175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:40.360175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:40.360175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:40.360175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 21:39:40.360175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:40.360175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 515/1038] 10.222.0.35 () {56 vars in 1093 bytes} [Fri May 13 21:39:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:40.367492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f361c39-6a6f-4c3b-bf89-af4f430458e9 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:40.471929 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f361c39-6a6f-4c3b-bf89-af4f430458e9 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:40.503997 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec847e-f451-4d11-958d-ae55fca88e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:40.504661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec847e-f451-4d11-958d-ae55fca88e5a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:40.504939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec847e-f451-4d11-958d-ae55fca88e5a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:40.505212 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec847e-f451-4d11-958d-ae55fca88e5a None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:40.506030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec847e-f451-4d11-958d-ae55fca88e5a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:40.506334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec847e-f451-4d11-958d-ae55fca88e5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:40.507835 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec847e-f451-4d11-958d-ae55fca88e5a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:40.529954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 524/1039] 10.222.0.35 () {56 vars in 1133 bytes} [Fri May 13 21:39:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:40.531527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363de90-a6da-49e7-9581-f21f0a27b3aa None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:40.629977 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363de90-a6da-49e7-9581-f21f0a27b3aa None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:40.662955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ea944-f5b6-449c-a527-75098c0b2c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:40.663601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ea944-f5b6-449c-a527-75098c0b2c68 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:40.663886 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ea944-f5b6-449c-a527-75098c0b2c68 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:40.664244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ea944-f5b6-449c-a527-75098c0b2c68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:40.729241 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ea944-f5b6-449c-a527-75098c0b2c68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:40.730717 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ea944-f5b6-449c-a527-75098c0b2c68 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:40.836590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 516/1040] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:39:40 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 21:39:40.838327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec847e-f451-4d11-958d-ae55fca88e5a None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:40.932223 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec847e-f451-4d11-958d-ae55fca88e5a None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:40.936003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a2b70-404d-4bb0-9ab3-3f9eecb4e26d None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:40.936291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a2b70-404d-4bb0-9ab3-3f9eecb4e26d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:40.936570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a2b70-404d-4bb0-9ab3-3f9eecb4e26d None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:40.938145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 517/1041] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:40.943925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740176c-4d0b-4315-8534-e7d196f5f36e None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:40.944234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740176c-4d0b-4315-8534-e7d196f5f36e None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:40.944488 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740176c-4d0b-4315-8534-e7d196f5f36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:40.962021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-f740176c-4d0b-4315-8534-e7d196f5f36e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:40.963243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f740176c-4d0b-4315-8534-e7d196f5f36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:40.970203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7faefa-422c-4776-b06a-dfe9bdfc3a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:40.971039 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7faefa-422c-4776-b06a-dfe9bdfc3a1b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:40.971318 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7faefa-422c-4776-b06a-dfe9bdfc3a1b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:40.971593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7faefa-422c-4776-b06a-dfe9bdfc3a1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:40.982156 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7faefa-422c-4776-b06a-dfe9bdfc3a1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:40.983643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7faefa-422c-4776-b06a-dfe9bdfc3a1b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-5c7faefa-422c-4776-b06a-dfe9bdfc3a1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:41.001089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:41.002254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:41.003207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:41.003207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:41.003207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:41.003207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:41.007042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 525/1042] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:41.008180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7faefa-422c-4776-b06a-dfe9bdfc3a1b None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:41.030781 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 518/1043] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:41.053680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740176c-4d0b-4315-8534-e7d196f5f36e None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:41.104314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7faefa-422c-4776-b06a-dfe9bdfc3a1b None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:41.137736 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20e278-f299-489b-a50f-164a061ead8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:41.138391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20e278-f299-489b-a50f-164a061ead8d None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:41.138672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20e278-f299-489b-a50f-164a061ead8d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:41.138974 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20e278-f299-489b-a50f-164a061ead8d None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:41.139683 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20e278-f299-489b-a50f-164a061ead8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:41.139988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20e278-f299-489b-a50f-164a061ead8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:41.141352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20e278-f299-489b-a50f-164a061ead8d None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:41.148459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740176c-4d0b-4315-8534-e7d196f5f36e None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:41.152223 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 526/1044] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:41.153564 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384aa94-7705-4a42-975a-cd9171fea883 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:41.153856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384aa94-7705-4a42-975a-cd9171fea883 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:41.154135 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384aa94-7705-4a42-975a-cd9171fea883 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:41.155535 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 527/1045] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:41.157909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384aa94-7705-4a42-975a-cd9171fea883 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:41.183164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75509b87-3aeb-4aa9-b87a-9c04b0f574c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:41.184574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75509b87-3aeb-4aa9-b87a-9c04b0f574c5 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:41.184859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75509b87-3aeb-4aa9-b87a-9c04b0f574c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:41.185130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75509b87-3aeb-4aa9-b87a-9c04b0f574c5 None admin] PATH_INFO: `/v3/services/image` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:41.185773 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75509b87-3aeb-4aa9-b87a-9c04b0f574c5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:41.187203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75509b87-3aeb-4aa9-b87a-9c04b0f574c5 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-75509b87-3aeb-4aa9-b87a-9c04b0f574c5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:41.198505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:41.199547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:41.200403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 519/1046] 10.222.0.35 () {56 vars in 1105 bytes} [Fri May 13 21:39:41 2022] GET /identity/v3/services/image => generated 86 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:41.202269 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ee7aa-2697-4db3-828f-e4f7dabf8eb2 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:41.202563 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ee7aa-2697-4db3-828f-e4f7dabf8eb2 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:41.202881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ee7aa-2697-4db3-828f-e4f7dabf8eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:41.218920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ba7ee7aa-2697-4db3-828f-e4f7dabf8eb2 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:41.220099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ba7ee7aa-2697-4db3-828f-e4f7dabf8eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:41.254108 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384aa94-7705-4a42-975a-cd9171fea883 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:41.282358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 520/1047] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:41.283207 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ee7aa-2697-4db3-828f-e4f7dabf8eb2 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:41.285845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b7c33-b18a-4978-87d2-1b874917978e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:41.286675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b7c33-b18a-4978-87d2-1b874917978e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:41.287004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b7c33-b18a-4978-87d2-1b874917978e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:41.287281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b7c33-b18a-4978-87d2-1b874917978e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:41.294478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b7c33-b18a-4978-87d2-1b874917978e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:41.295947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b7c33-b18a-4978-87d2-1b874917978e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-7a7b7c33-b18a-4978-87d2-1b874917978e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:41.310502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:41.311570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:41.312537 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:41.312537 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:41.312537 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:41.312537 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:41.312537 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:41.312537 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:41.312537 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:41.312537 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:41.316286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 528/1048] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:41.317086 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b7c33-b18a-4978-87d2-1b874917978e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:41.375352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ee7aa-2697-4db3-828f-e4f7dabf8eb2 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:41.406919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5533f1-6503-47c7-b501-ff4674d95fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:41.407566 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5533f1-6503-47c7-b501-ff4674d95fb7 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:41.407844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5533f1-6503-47c7-b501-ff4674d95fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:41.408118 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5533f1-6503-47c7-b501-ff4674d95fb7 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:41.408537 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b7c33-b18a-4978-87d2-1b874917978e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:41.408809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5533f1-6503-47c7-b501-ff4674d95fb7 None admin] RBAC: Adding query filter params (name=image) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:41.409109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5533f1-6503-47c7-b501-ff4674d95fb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:41.410471 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5533f1-6503-47c7-b501-ff4674d95fb7 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:41.415272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 521/1049] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:41.415848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5533f1-6503-47c7-b501-ff4674d95fb7 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:41.445623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbb534-b020-446d-965b-fbfbb40ba41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:41.446342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbb534-b020-446d-965b-fbfbb40ba41a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:41.446622 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbb534-b020-446d-965b-fbfbb40ba41a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:41.446936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbb534-b020-446d-965b-fbfbb40ba41a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:41.457760 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbb534-b020-446d-965b-fbfbb40ba41a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-aacbb534-b020-446d-965b-fbfbb40ba41a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:41.466442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:41.467455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:41.468301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:41.468301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:41.468301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:41.468301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:41.468301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:41.468301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:41.468301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:41.468301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:41.468301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:41.468301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:41.468301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:41.468301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:41.468301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 529/1050] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:39:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:41.469578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbb534-b020-446d-965b-fbfbb40ba41a None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:41.513494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5533f1-6503-47c7-b501-ff4674d95fb7 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:41.545830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a1e1f-bcfe-45d5-9d96-4f2e8b355d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:41.546475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a1e1f-bcfe-45d5-9d96-4f2e8b355d4f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:41.546786 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a1e1f-bcfe-45d5-9d96-4f2e8b355d4f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:41.547062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a1e1f-bcfe-45d5-9d96-4f2e8b355d4f None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:41.547829 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a1e1f-bcfe-45d5-9d96-4f2e8b355d4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:41.548133 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a1e1f-bcfe-45d5-9d96-4f2e8b355d4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:41.549558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a1e1f-bcfe-45d5-9d96-4f2e8b355d4f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:41.560911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 522/1051] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:41.561692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a1e1f-bcfe-45d5-9d96-4f2e8b355d4f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:41.561818 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbb534-b020-446d-965b-fbfbb40ba41a None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:41.594083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f388fd-ea06-4a74-876a-c39faceeaaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:41.594759 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f388fd-ea06-4a74-876a-c39faceeaaf6 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:41.595047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f388fd-ea06-4a74-876a-c39faceeaaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:41.595324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f388fd-ea06-4a74-876a-c39faceeaaf6 None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:41.596020 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f388fd-ea06-4a74-876a-c39faceeaaf6 None admin] RBAC: Adding query filter params (type=image) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:41.596323 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f388fd-ea06-4a74-876a-c39faceeaaf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:41.597690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f388fd-ea06-4a74-876a-c39faceeaaf6 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:41.602425 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 530/1052] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:41.603142 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f388fd-ea06-4a74-876a-c39faceeaaf6 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:41.660645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a1e1f-bcfe-45d5-9d96-4f2e8b355d4f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:41.692445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8caf1-c6f0-4a40-8b17-5f54c3a24329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:41.693092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8caf1-c6f0-4a40-8b17-5f54c3a24329 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:41.693368 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8caf1-c6f0-4a40-8b17-5f54c3a24329 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:41.693647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8caf1-c6f0-4a40-8b17-5f54c3a24329 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:41.694373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8caf1-c6f0-4a40-8b17-5f54c3a24329 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:41.694676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8caf1-c6f0-4a40-8b17-5f54c3a24329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:41.695684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f388fd-ea06-4a74-876a-c39faceeaaf6 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:41.696062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8caf1-c6f0-4a40-8b17-5f54c3a24329 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:41.704275 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 523/1053] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:39:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:41.705014 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8caf1-c6f0-4a40-8b17-5f54c3a24329 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:41.729350 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1ffd8-12e8-46a6-8798-24f0aa5e5ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:41.731357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1ffd8-12e8-46a6-8798-24f0aa5e5ebb None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:41.731645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1ffd8-12e8-46a6-8798-24f0aa5e5ebb None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:41.731925 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1ffd8-12e8-46a6-8798-24f0aa5e5ebb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:41.732749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1ffd8-12e8-46a6-8798-24f0aa5e5ebb None admin] RBAC: Adding query filter params (role.id=708710710e3e44dca3a6d11cbc2b6dd6, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=316ef43c6c304bd3a27e085bf4da8a5e) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:41.733061 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1ffd8-12e8-46a6-8798-24f0aa5e5ebb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:41.736367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1ffd8-12e8-46a6-8798-24f0aa5e5ebb None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:41.742199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 531/1054] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:39:41 2022] GET /identity/v3/role_assignments?user.id=316ef43c6c304bd3a27e085bf4da8a5e&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=708710710e3e44dca3a6d11cbc2b6dd6 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:41.743004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1ffd8-12e8-46a6-8798-24f0aa5e5ebb None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:41.826893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8caf1-c6f0-4a40-8b17-5f54c3a24329 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:41.847705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1ffd8-12e8-46a6-8798-24f0aa5e5ebb None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:41.862161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021c0c0-ada8-44e0-b32f-d0ea7b613772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:41.862917 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021c0c0-ada8-44e0-b32f-d0ea7b613772 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:41.863200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021c0c0-ada8-44e0-b32f-d0ea7b613772 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:41.863481 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021c0c0-ada8-44e0-b32f-d0ea7b613772 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:41.864285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021c0c0-ada8-44e0-b32f-d0ea7b613772 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:41.865682 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021c0c0-ada8-44e0-b32f-d0ea7b613772 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:41.883006 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9893bff-5305-4d60-8172-b2d21b9d1332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:41.883896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9893bff-5305-4d60-8172-b2d21b9d1332 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:41.884181 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9893bff-5305-4d60-8172-b2d21b9d1332 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:41.884460 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9893bff-5305-4d60-8172-b2d21b9d1332 None admin] PATH_INFO: `/v3/users/60b9b7d7631741439900b296725ce77f` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:41.886643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 524/1055] 10.222.0.35 () {60 vars in 1144 bytes} [Fri May 13 21:39:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:39:41.901197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9893bff-5305-4d60-8172-b2d21b9d1332 None admin] RBAC: Authorizing `identity:get_user(user_id=60b9b7d7631741439900b296725ce77f)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:41.903512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9893bff-5305-4d60-8172-b2d21b9d1332 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:41.905718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 532/1056] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:39:41 2022] GET /identity/v3/users/60b9b7d7631741439900b296725ce77f => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:41.911820 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021c0c0-ada8-44e0-b32f-d0ea7b613772 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:42.025959 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021c0c0-ada8-44e0-b32f-d0ea7b613772 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:42.067964 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89fdd9-27e1-48a0-aae9-8750522c94be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:42.068627 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89fdd9-27e1-48a0-aae9-8750522c94be None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:42.068905 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89fdd9-27e1-48a0-aae9-8750522c94be None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:42.069174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89fdd9-27e1-48a0-aae9-8750522c94be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:42.137198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89fdd9-27e1-48a0-aae9-8750522c94be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:42.139218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89fdd9-27e1-48a0-aae9-8750522c94be None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:42.239268 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 525/1057] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:39:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:39:42.240855 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9893bff-5305-4d60-8172-b2d21b9d1332 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:42.350338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9893bff-5305-4d60-8172-b2d21b9d1332 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:42.382888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f959730-3ce7-415d-89de-56dfbe8df8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:42.383736 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f959730-3ce7-415d-89de-56dfbe8df8fc None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:42.384016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f959730-3ce7-415d-89de-56dfbe8df8fc None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:42.384290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f959730-3ce7-415d-89de-56dfbe8df8fc None admin] PATH_INFO: `/v3/projects/b54c31fc82b24b52be69f22d41907044` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:42.385453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f959730-3ce7-415d-89de-56dfbe8df8fc None admin] RBAC: Authorizing `identity:get_project(project_id=b54c31fc82b24b52be69f22d41907044)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:42.386915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f959730-3ce7-415d-89de-56dfbe8df8fc None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:42.393947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 533/1058] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:39:42 2022] GET /identity/v3/projects/b54c31fc82b24b52be69f22d41907044 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:42.395448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89fdd9-27e1-48a0-aae9-8750522c94be None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:42.493759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89fdd9-27e1-48a0-aae9-8750522c94be None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:42.526428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d565acc-1d5f-437e-9f3d-2ec6cbdf5463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:42.527110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d565acc-1d5f-437e-9f3d-2ec6cbdf5463 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:42.527389 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d565acc-1d5f-437e-9f3d-2ec6cbdf5463 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:42.527694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d565acc-1d5f-437e-9f3d-2ec6cbdf5463 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:42.528509 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d565acc-1d5f-437e-9f3d-2ec6cbdf5463 None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=b54c31fc82b24b52be69f22d41907044, user.id=60b9b7d7631741439900b296725ce77f) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:42.528819 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d565acc-1d5f-437e-9f3d-2ec6cbdf5463 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:42.530249 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d565acc-1d5f-437e-9f3d-2ec6cbdf5463 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:42.535835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 526/1059] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:39:42 2022] GET /identity/v3/role_assignments?user.id=60b9b7d7631741439900b296725ce77f&scope.project.id=b54c31fc82b24b52be69f22d41907044&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:44.002086 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd988a4-a03c-4f33-89b2-53169ee32ece None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:44.002609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd988a4-a03c-4f33-89b2-53169ee32ece None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:44.002609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd988a4-a03c-4f33-89b2-53169ee32ece None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:44.004077 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 534/1060] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:44.009135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9c452-ff3b-40b3-90a6-a66c6336bd43 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:44.009554 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9c452-ff3b-40b3-90a6-a66c6336bd43 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:44.009697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9c452-ff3b-40b3-90a6-a66c6336bd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:44.026682 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-13f9c452-ff3b-40b3-90a6-a66c6336bd43 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:44.028526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-13f9c452-ff3b-40b3-90a6-a66c6336bd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:44.092862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 527/1061] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:44.115401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd988a4-a03c-4f33-89b2-53169ee32ece None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:44.134338 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1057d-d191-4800-914b-c1ca93b1b8e2 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:44.134628 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1057d-d191-4800-914b-c1ca93b1b8e2 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:44.134945 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1057d-d191-4800-914b-c1ca93b1b8e2 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:44.136446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 528/1062] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:39:44.141973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f2ed9-4ffe-4bfd-b997-a7be34f10f66 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:44.142264 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f2ed9-4ffe-4bfd-b997-a7be34f10f66 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:44.142547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f2ed9-4ffe-4bfd-b997-a7be34f10f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:44.160870 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ae3f2ed9-4ffe-4bfd-b997-a7be34f10f66 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:44.162371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ae3f2ed9-4ffe-4bfd-b997-a7be34f10f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:44.224959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd988a4-a03c-4f33-89b2-53169ee32ece None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:44.234482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 529/1063] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:44.257381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f2ed9-4ffe-4bfd-b997-a7be34f10f66 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:44.257532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c934e-2ce1-4b8e-941d-900c3ff8000f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:44.258210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c934e-2ce1-4b8e-941d-900c3ff8000f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:44.258493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c934e-2ce1-4b8e-941d-900c3ff8000f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:44.258808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c934e-2ce1-4b8e-941d-900c3ff8000f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:44.269516 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c934e-2ce1-4b8e-941d-900c3ff8000f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-181c934e-2ce1-4b8e-941d-900c3ff8000f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:44.277645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:44.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:44.279970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:44.279970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:44.279970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:44.279970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:44.279970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:44.279970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:44.279970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:44.279970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:44.279970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:44.279970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:44.279970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 21:39:44.279970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:44.279970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 535/1064] 10.222.0.35 () {56 vars in 1120 bytes} [Fri May 13 21:39:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:44.285414 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c934e-2ce1-4b8e-941d-900c3ff8000f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:44.358295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f2ed9-4ffe-4bfd-b997-a7be34f10f66 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:44.382280 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c934e-2ce1-4b8e-941d-900c3ff8000f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:44.391999 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477149df-ac4e-4bdf-b565-df4f3d7c0235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:44.392647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477149df-ac4e-4bdf-b565-df4f3d7c0235 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:44.392938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477149df-ac4e-4bdf-b565-df4f3d7c0235 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:44.393237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477149df-ac4e-4bdf-b565-df4f3d7c0235 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:44.416247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699c492-10d0-4623-97f3-224486333f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:44.417153 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699c492-10d0-4623-97f3-224486333f20 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:44.417433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699c492-10d0-4623-97f3-224486333f20 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:44.417710 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699c492-10d0-4623-97f3-224486333f20 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:44.418472 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699c492-10d0-4623-97f3-224486333f20 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:44.418806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699c492-10d0-4623-97f3-224486333f20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:44.420185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699c492-10d0-4623-97f3-224486333f20 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:44.428162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 536/1065] 10.222.0.35 () {56 vars in 1115 bytes} [Fri May 13 21:39:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:44.433547 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699c492-10d0-4623-97f3-224486333f20 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:44.460161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477149df-ac4e-4bdf-b565-df4f3d7c0235 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:44.461620 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477149df-ac4e-4bdf-b565-df4f3d7c0235 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:44.529981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699c492-10d0-4623-97f3-224486333f20 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:44.560189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 530/1066] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:39:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 21:39:44.566561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477149df-ac4e-4bdf-b565-df4f3d7c0235 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:44.575326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415440c-fc44-4e36-8c5b-0bda518f7906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:44.576147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415440c-fc44-4e36-8c5b-0bda518f7906 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:44.576428 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415440c-fc44-4e36-8c5b-0bda518f7906 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:44.576701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415440c-fc44-4e36-8c5b-0bda518f7906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:44.588594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415440c-fc44-4e36-8c5b-0bda518f7906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:44.590129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415440c-fc44-4e36-8c5b-0bda518f7906 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-3415440c-fc44-4e36-8c5b-0bda518f7906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:44.608045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:44.609004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:44.614927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:44.614927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:44.614927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:44.614927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:44.614927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 537/1067] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:44.615433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415440c-fc44-4e36-8c5b-0bda518f7906 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:44.667218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477149df-ac4e-4bdf-b565-df4f3d7c0235 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:44.701028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854835b-8292-4fa8-8a44-073b4880c2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:44.701833 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854835b-8292-4fa8-8a44-073b4880c2c0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:44.702114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854835b-8292-4fa8-8a44-073b4880c2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:44.702388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854835b-8292-4fa8-8a44-073b4880c2c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:44.712695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854835b-8292-4fa8-8a44-073b4880c2c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:44.714158 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854835b-8292-4fa8-8a44-073b4880c2c0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:44.715836 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415440c-fc44-4e36-8c5b-0bda518f7906 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-0854835b-8292-4fa8-8a44-073b4880c2c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:44.732201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:44.733130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:44.734055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:44.734055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:44.734055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:44.734055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:44.734055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 531/1068] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:44.739160 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854835b-8292-4fa8-8a44-073b4880c2c0 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:44.752097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398ff8e-c6c7-499b-a235-49a6d5cbf33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:44.752750 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398ff8e-c6c7-499b-a235-49a6d5cbf33d None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:44.753031 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398ff8e-c6c7-499b-a235-49a6d5cbf33d None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:44.753304 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398ff8e-c6c7-499b-a235-49a6d5cbf33d None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:44.754011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398ff8e-c6c7-499b-a235-49a6d5cbf33d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:44.754315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398ff8e-c6c7-499b-a235-49a6d5cbf33d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:44.755710 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398ff8e-c6c7-499b-a235-49a6d5cbf33d None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:44.767496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 538/1069] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:44.768747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33292b-75b7-43be-a9f0-6aea338f4993 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:44.769033 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33292b-75b7-43be-a9f0-6aea338f4993 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:44.769339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33292b-75b7-43be-a9f0-6aea338f4993 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:44.770643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 539/1070] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:44.772548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33292b-75b7-43be-a9f0-6aea338f4993 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:44.838877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854835b-8292-4fa8-8a44-073b4880c2c0 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:44.867054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33292b-75b7-43be-a9f0-6aea338f4993 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:44.870929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81caff-4a01-467c-8c74-caafce757d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:44.872219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81caff-4a01-467c-8c74-caafce757d38 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:44.872503 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81caff-4a01-467c-8c74-caafce757d38 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:44.872782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81caff-4a01-467c-8c74-caafce757d38 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:44.873499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81caff-4a01-467c-8c74-caafce757d38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:44.873803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81caff-4a01-467c-8c74-caafce757d38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:44.875208 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81caff-4a01-467c-8c74-caafce757d38 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:44.886284 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 532/1071] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:44.887514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44ebae-fafc-43c7-b4ad-92c2cfe6c7e9 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:44.887807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44ebae-fafc-43c7-b4ad-92c2cfe6c7e9 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:44.888085 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44ebae-fafc-43c7-b4ad-92c2cfe6c7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:44.900320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b143b-a425-421c-946b-f10f82466e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:44.901135 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b143b-a425-421c-946b-f10f82466e96 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:44.901466 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b143b-a425-421c-946b-f10f82466e96 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:44.901747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b143b-a425-421c-946b-f10f82466e96 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:44.906214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-9e44ebae-fafc-43c7-b4ad-92c2cfe6c7e9 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:44.907494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-9e44ebae-fafc-43c7-b4ad-92c2cfe6c7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:44.910045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b143b-a425-421c-946b-f10f82466e96 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:44.911576 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b143b-a425-421c-946b-f10f82466e96 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-259b143b-a425-421c-946b-f10f82466e96 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:44.927661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:44.928916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:44.930109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:44.930109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:44.930109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:44.930109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:44.930109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:44.930109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:44.930109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:44.930109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:44.930109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 21:39:44.930109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:44.934457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 540/1072] 10.222.0.35 () {56 vars in 1093 bytes} [Fri May 13 21:39:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:44.935434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b143b-a425-421c-946b-f10f82466e96 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:44.981419 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 533/1073] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:45.004734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44ebae-fafc-43c7-b4ad-92c2cfe6c7e9 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:45.037117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b143b-a425-421c-946b-f10f82466e96 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:45.072947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123bafa-f1cd-44cb-b45e-953df1b92cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:45.073617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123bafa-f1cd-44cb-b45e-953df1b92cf3 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:45.073896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123bafa-f1cd-44cb-b45e-953df1b92cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:45.074165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123bafa-f1cd-44cb-b45e-953df1b92cf3 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:45.075072 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123bafa-f1cd-44cb-b45e-953df1b92cf3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:45.075384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123bafa-f1cd-44cb-b45e-953df1b92cf3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:45.076807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123bafa-f1cd-44cb-b45e-953df1b92cf3 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:45.095279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 541/1074] 10.222.0.35 () {56 vars in 1133 bytes} [Fri May 13 21:39:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:45.101093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123bafa-f1cd-44cb-b45e-953df1b92cf3 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:45.106816 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44ebae-fafc-43c7-b4ad-92c2cfe6c7e9 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:45.141162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2179c-79a9-41e9-8425-3159152a45f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:45.141956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2179c-79a9-41e9-8425-3159152a45f2 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:45.142236 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2179c-79a9-41e9-8425-3159152a45f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:45.142508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2179c-79a9-41e9-8425-3159152a45f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:45.153448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2179c-79a9-41e9-8425-3159152a45f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-19f2179c-79a9-41e9-8425-3159152a45f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:45.161121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:45.162037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:45.162971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:45.162971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:45.162971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:45.162971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:45.162971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:45.162971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:45.162971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:45.162971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:45.162971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:45.162971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:45.162971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:45.162971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:45.162971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 534/1075] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:39:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:45.168873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2179c-79a9-41e9-8425-3159152a45f2 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:45.214795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123bafa-f1cd-44cb-b45e-953df1b92cf3 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:45.247447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31558946-4f65-45a0-bdba-35602b352a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:45.248225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31558946-4f65-45a0-bdba-35602b352a05 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:45.248505 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31558946-4f65-45a0-bdba-35602b352a05 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:45.248776 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31558946-4f65-45a0-bdba-35602b352a05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:45.258556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31558946-4f65-45a0-bdba-35602b352a05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:45.260038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31558946-4f65-45a0-bdba-35602b352a05 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:45.264510 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2179c-79a9-41e9-8425-3159152a45f2 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-31558946-4f65-45a0-bdba-35602b352a05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:45.276635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:45.278041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:45.278964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:45.278964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:45.278964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:45.278964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:45.278964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 542/1076] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:45.283895 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31558946-4f65-45a0-bdba-35602b352a05 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:45.297516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b568bda-55b2-4990-8e93-65898f41a392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:45.298176 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b568bda-55b2-4990-8e93-65898f41a392 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:45.298460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b568bda-55b2-4990-8e93-65898f41a392 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:45.298776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b568bda-55b2-4990-8e93-65898f41a392 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:45.299515 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b568bda-55b2-4990-8e93-65898f41a392 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:45.299824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b568bda-55b2-4990-8e93-65898f41a392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:45.301209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b568bda-55b2-4990-8e93-65898f41a392 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:45.309390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 535/1077] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:39:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:45.314982 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b568bda-55b2-4990-8e93-65898f41a392 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:45.376686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31558946-4f65-45a0-bdba-35602b352a05 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:45.408409 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b568bda-55b2-4990-8e93-65898f41a392 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:45.408557 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4129a5-a4d7-45a8-b70f-92fbb4a70810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:45.409081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4129a5-a4d7-45a8-b70f-92fbb4a70810 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:45.409362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4129a5-a4d7-45a8-b70f-92fbb4a70810 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:45.409652 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4129a5-a4d7-45a8-b70f-92fbb4a70810 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:45.410342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4129a5-a4d7-45a8-b70f-92fbb4a70810 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:45.410643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4129a5-a4d7-45a8-b70f-92fbb4a70810 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:45.412031 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4129a5-a4d7-45a8-b70f-92fbb4a70810 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:45.423190 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 543/1078] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:45.428581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4129a5-a4d7-45a8-b70f-92fbb4a70810 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:45.440732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e9ff0-1142-4c95-9b05-78b62ec3a4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:45.441595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e9ff0-1142-4c95-9b05-78b62ec3a4ea None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:45.441881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e9ff0-1142-4c95-9b05-78b62ec3a4ea None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:45.442156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e9ff0-1142-4c95-9b05-78b62ec3a4ea None admin] PATH_INFO: `/v3/users/60b9b7d7631741439900b296725ce77f` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:45.443367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e9ff0-1142-4c95-9b05-78b62ec3a4ea None admin] RBAC: Authorizing `identity:get_user(user_id=60b9b7d7631741439900b296725ce77f)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:45.444792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e9ff0-1142-4c95-9b05-78b62ec3a4ea None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:45.446553 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 536/1079] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:39:45 2022] GET /identity/v3/users/60b9b7d7631741439900b296725ce77f => generated 285 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:45.452336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e9ff0-1142-4c95-9b05-78b62ec3a4ea None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:45.521861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4129a5-a4d7-45a8-b70f-92fbb4a70810 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:45.544682 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e9ff0-1142-4c95-9b05-78b62ec3a4ea None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:45.553386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a58fd5-f72d-4762-a0ab-e1c45aa4c3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:45.554160 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a58fd5-f72d-4762-a0ab-e1c45aa4c3cf None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:45.554438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a58fd5-f72d-4762-a0ab-e1c45aa4c3cf None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:45.554729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a58fd5-f72d-4762-a0ab-e1c45aa4c3cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:45.561407 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a58fd5-f72d-4762-a0ab-e1c45aa4c3cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:45.562872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a58fd5-f72d-4762-a0ab-e1c45aa4c3cf None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:45.576426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574568f-4384-4b47-9267-07b9c853b6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-74a58fd5-f72d-4762-a0ab-e1c45aa4c3cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:45.577109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:45.578103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:45.579030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:45.579030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:45.579030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:45.579030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:45.579030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:45.579030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:45.579030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:45.579030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:45.579468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574568f-4384-4b47-9267-07b9c853b6f1 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:45.579468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574568f-4384-4b47-9267-07b9c853b6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:45.579468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574568f-4384-4b47-9267-07b9c853b6f1 None admin] PATH_INFO: `/v3/projects/b54c31fc82b24b52be69f22d41907044` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:45.579652 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 544/1080] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:45.579728 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0574568f-4384-4b47-9267-07b9c853b6f1 None admin] RBAC: Authorizing `identity:get_project(project_id=b54c31fc82b24b52be69f22d41907044)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:45.580650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0574568f-4384-4b47-9267-07b9c853b6f1 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:45.582350 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 537/1081] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:39:45 2022] GET /identity/v3/projects/b54c31fc82b24b52be69f22d41907044 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:45.583949 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a58fd5-f72d-4762-a0ab-e1c45aa4c3cf None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:45.587831 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574568f-4384-4b47-9267-07b9c853b6f1 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:45.676031 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a58fd5-f72d-4762-a0ab-e1c45aa4c3cf None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:45.679881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574568f-4384-4b47-9267-07b9c853b6f1 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:45.707942 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ed495-86f3-4e07-81d4-859be103eb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:45.708580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ed495-86f3-4e07-81d4-859be103eb98 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:45.708863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ed495-86f3-4e07-81d4-859be103eb98 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:45.709136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ed495-86f3-4e07-81d4-859be103eb98 None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:45.709906 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ed495-86f3-4e07-81d4-859be103eb98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:45.710212 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ed495-86f3-4e07-81d4-859be103eb98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:45.711241 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f085b-0aec-4d24-8748-81185e5af06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:45.711675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ed495-86f3-4e07-81d4-859be103eb98 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:45.712101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f085b-0aec-4d24-8748-81185e5af06d None admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:45.712379 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f085b-0aec-4d24-8748-81185e5af06d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:45.712652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f085b-0aec-4d24-8748-81185e5af06d None admin] PATH_INFO: `/v3/projects/b54c31fc82b24b52be69f22d41907044/users/60b9b7d7631741439900b296725ce77f/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:45.714590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f085b-0aec-4d24-8748-81185e5af06d None admin] RBAC: Authorizing `identity:create_grant(project_id=b54c31fc82b24b52be69f22d41907044, user_id=60b9b7d7631741439900b296725ce77f, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:45.716704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f085b-0aec-4d24-8748-81185e5af06d None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:45.724008 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 545/1082] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:45.730127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ed495-86f3-4e07-81d4-859be103eb98 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:45.732223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 538/1083] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:39:45 2022] PUT /identity/v3/projects/b54c31fc82b24b52be69f22d41907044/users/60b9b7d7631741439900b296725ce77f/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:39:45.832970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ed495-86f3-4e07-81d4-859be103eb98 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:45.865793 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd4508-0377-4d3b-b92e-efa743f291e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:45.866681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd4508-0377-4d3b-b92e-efa743f291e4 None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:45.866987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd4508-0377-4d3b-b92e-efa743f291e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:45.867260 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd4508-0377-4d3b-b92e-efa743f291e4 None admin] PATH_INFO: `/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/316ef43c6c304bd3a27e085bf4da8a5e/roles/708710710e3e44dca3a6d11cbc2b6dd6` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:45.869484 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd4508-0377-4d3b-b92e-efa743f291e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac834e84b2c84261ade311a6dfbbeb20, user_id=316ef43c6c304bd3a27e085bf4da8a5e, role_id=708710710e3e44dca3a6d11cbc2b6dd6)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:45.870951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd4508-0377-4d3b-b92e-efa743f291e4 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:45.885171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 546/1084] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:39:45 2022] PUT /identity/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/316ef43c6c304bd3a27e085bf4da8a5e/roles/708710710e3e44dca3a6d11cbc2b6dd6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:39:47.944056 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b1f11-9422-45a0-a456-a330561405cf None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:47.944586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b1f11-9422-45a0-a456-a330561405cf None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:47.944748 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b1f11-9422-45a0-a456-a330561405cf None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:47.945998 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 539/1085] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:47.951872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831c76f-61a5-4129-b2ec-9bbdec85ccc6 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:47.952274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831c76f-61a5-4129-b2ec-9bbdec85ccc6 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:47.952424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831c76f-61a5-4129-b2ec-9bbdec85ccc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:47.969466 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a831c76f-61a5-4129-b2ec-9bbdec85ccc6 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:47.970643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a831c76f-61a5-4129-b2ec-9bbdec85ccc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:48.033213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 547/1086] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:48.056390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b1f11-9422-45a0-a456-a330561405cf None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:48.078003 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7962f-cec3-4495-91bb-465eac3baefb None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:48.078289 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7962f-cec3-4495-91bb-465eac3baefb None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:48.078563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7962f-cec3-4495-91bb-465eac3baefb None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:48.079958 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 548/1087] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:39:48.084792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80859129-4d85-4565-b711-90b9111ba062 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:48.085074 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80859129-4d85-4565-b711-90b9111ba062 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:48.085350 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80859129-4d85-4565-b711-90b9111ba062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:48.101107 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-80859129-4d85-4565-b711-90b9111ba062 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:48.102275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-80859129-4d85-4565-b711-90b9111ba062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:48.155677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b1f11-9422-45a0-a456-a330561405cf None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:48.164805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 549/1088] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:39:48.187909 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e246a5f-64d6-41eb-a206-ee193706ebed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:48.188579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80859129-4d85-4565-b711-90b9111ba062 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:48.188706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e246a5f-64d6-41eb-a206-ee193706ebed None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:48.188969 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e246a5f-64d6-41eb-a206-ee193706ebed None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:48.189245 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e246a5f-64d6-41eb-a206-ee193706ebed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:48.200652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e246a5f-64d6-41eb-a206-ee193706ebed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-3e246a5f-64d6-41eb-a206-ee193706ebed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:48.208052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:48.208993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:48.209841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:48.209841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:48.209841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:48.209841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:48.209841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:48.209841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:48.209841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:48.209841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:48.209841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:48.209841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:48.209841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:39:48.209841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:39:48.209841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 540/1089] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:39:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:48.215013 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e246a5f-64d6-41eb-a206-ee193706ebed None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:48.281624 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80859129-4d85-4565-b711-90b9111ba062 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:48.305801 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e246a5f-64d6-41eb-a206-ee193706ebed None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:48.313297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033b4cc-0568-4cb2-8d89-f283fc1694bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:48.314091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033b4cc-0568-4cb2-8d89-f283fc1694bb None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:48.314370 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033b4cc-0568-4cb2-8d89-f283fc1694bb None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:48.314640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033b4cc-0568-4cb2-8d89-f283fc1694bb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:48.324288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033b4cc-0568-4cb2-8d89-f283fc1694bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-7033b4cc-0568-4cb2-8d89-f283fc1694bb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:39:48.332738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:48.333863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:48.334754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:48.334754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:48.334754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:48.334754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:48.334754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:39:48.334754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:39:48.334754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:39:48.334754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:39:48.334754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:39:48.334754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:39:48.334754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 21:39:48.334754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:48.334754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 550/1090] 10.222.0.35 () {56 vars in 1120 bytes} [Fri May 13 21:39:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:48.338980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34e6f5-2ee8-427b-8a6c-def5e187c586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:48.339633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34e6f5-2ee8-427b-8a6c-def5e187c586 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:48.339786 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033b4cc-0568-4cb2-8d89-f283fc1694bb None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:48.339935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34e6f5-2ee8-427b-8a6c-def5e187c586 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:48.340208 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34e6f5-2ee8-427b-8a6c-def5e187c586 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:48.340940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34e6f5-2ee8-427b-8a6c-def5e187c586 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:48.341240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34e6f5-2ee8-427b-8a6c-def5e187c586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:48.342611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34e6f5-2ee8-427b-8a6c-def5e187c586 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:48.355010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 541/1091] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:39:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:48.355979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34e6f5-2ee8-427b-8a6c-def5e187c586 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:48.433522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033b4cc-0568-4cb2-8d89-f283fc1694bb None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:48.450826 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34e6f5-2ee8-427b-8a6c-def5e187c586 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:48.465542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a85e1-97fc-4112-ada2-563f72bcdd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:48.466222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a85e1-97fc-4112-ada2-563f72bcdd25 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:48.466502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a85e1-97fc-4112-ada2-563f72bcdd25 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:48.466806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a85e1-97fc-4112-ada2-563f72bcdd25 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:48.467531 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a85e1-97fc-4112-ada2-563f72bcdd25 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:48.467831 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a85e1-97fc-4112-ada2-563f72bcdd25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:48.469182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a85e1-97fc-4112-ada2-563f72bcdd25 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:48.476825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 551/1092] 10.222.0.35 () {56 vars in 1115 bytes} [Fri May 13 21:39:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:48.482785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a85e1-97fc-4112-ada2-563f72bcdd25 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:48.484510 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a1e4e-36f3-4ef0-8672-6a2a36fa5aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:48.485367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a1e4e-36f3-4ef0-8672-6a2a36fa5aea None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:48.485647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a1e4e-36f3-4ef0-8672-6a2a36fa5aea None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:48.485922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a1e4e-36f3-4ef0-8672-6a2a36fa5aea None admin] PATH_INFO: `/v3/users/60b9b7d7631741439900b296725ce77f` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:48.487163 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a1e4e-36f3-4ef0-8672-6a2a36fa5aea None admin] RBAC: Authorizing `identity:get_user(user_id=60b9b7d7631741439900b296725ce77f)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:48.488586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a1e4e-36f3-4ef0-8672-6a2a36fa5aea None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:48.494681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 542/1093] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:39:48 2022] GET /identity/v3/users/60b9b7d7631741439900b296725ce77f => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:48.495573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a1e4e-36f3-4ef0-8672-6a2a36fa5aea None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:48.575848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a85e1-97fc-4112-ada2-563f72bcdd25 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:48.589043 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a1e4e-36f3-4ef0-8672-6a2a36fa5aea None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:48.607738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225d267-d2da-4cf5-8747-bb260e876dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:48.608519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225d267-d2da-4cf5-8747-bb260e876dd1 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:48.608800 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225d267-d2da-4cf5-8747-bb260e876dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:48.609076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225d267-d2da-4cf5-8747-bb260e876dd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:48.619633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225d267-d2da-4cf5-8747-bb260e876dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:48.621060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225d267-d2da-4cf5-8747-bb260e876dd1 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:48.621223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73965c1-6ad3-42b0-b8a7-1c1ea41f2ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:48.622359 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73965c1-6ad3-42b0-b8a7-1c1ea41f2ee7 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:48.622639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73965c1-6ad3-42b0-b8a7-1c1ea41f2ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:48.622942 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73965c1-6ad3-42b0-b8a7-1c1ea41f2ee7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-c225d267-d2da-4cf5-8747-bb260e876dd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:48.638445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:48.639486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:48.640601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:48.640601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:48.640601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:48.640601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:48.640601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 552/1094] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:48.645232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225d267-d2da-4cf5-8747-bb260e876dd1 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:48.685834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73965c1-6ad3-42b0-b8a7-1c1ea41f2ee7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:48.687308 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73965c1-6ad3-42b0-b8a7-1c1ea41f2ee7 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:48.737151 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225d267-d2da-4cf5-8747-bb260e876dd1 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:48.769008 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e77c12-6798-4e5c-b4e0-a5542d2a218b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:48.769684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e77c12-6798-4e5c-b4e0-a5542d2a218b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:48.769964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e77c12-6798-4e5c-b4e0-a5542d2a218b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:48.770241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e77c12-6798-4e5c-b4e0-a5542d2a218b None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:48.770993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e77c12-6798-4e5c-b4e0-a5542d2a218b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:48.771300 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e77c12-6798-4e5c-b4e0-a5542d2a218b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:48.772667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e77c12-6798-4e5c-b4e0-a5542d2a218b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:48.779409 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 543/1095] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:39:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:39:48.784618 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 553/1096] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:48.785147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73965c1-6ad3-42b0-b8a7-1c1ea41f2ee7 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:48.789784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e77c12-6798-4e5c-b4e0-a5542d2a218b None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:48.883360 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e77c12-6798-4e5c-b4e0-a5542d2a218b None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:48.883648 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73965c1-6ad3-42b0-b8a7-1c1ea41f2ee7 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:48.915557 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ff9bf-e2a7-4717-b309-c44355e16ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:48.915756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0de5e1-088a-4a89-bdb0-a4a3be144e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:48.916399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ff9bf-e2a7-4717-b309-c44355e16ae0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:48.916498 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0de5e1-088a-4a89-bdb0-a4a3be144e44 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:48.916678 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ff9bf-e2a7-4717-b309-c44355e16ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:48.916777 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0de5e1-088a-4a89-bdb0-a4a3be144e44 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:48.916952 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ff9bf-e2a7-4717-b309-c44355e16ae0 None admin] PATH_INFO: `/v3/projects/b54c31fc82b24b52be69f22d41907044` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:48.917051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0de5e1-088a-4a89-bdb0-a4a3be144e44 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:48.918132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ff9bf-e2a7-4717-b309-c44355e16ae0 None admin] RBAC: Authorizing `identity:get_project(project_id=b54c31fc82b24b52be69f22d41907044)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:48.919600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ff9bf-e2a7-4717-b309-c44355e16ae0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:48.921451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 544/1097] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:39:48 2022] GET /identity/v3/projects/b54c31fc82b24b52be69f22d41907044 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:48.923886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0de5e1-088a-4a89-bdb0-a4a3be144e44 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:48.925341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0de5e1-088a-4a89-bdb0-a4a3be144e44 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:48.927301 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ff9bf-e2a7-4717-b309-c44355e16ae0 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-bc0de5e1-088a-4a89-bdb0-a4a3be144e44 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:39:48.940126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:48.941213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:48.942203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:48.942203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:48.942203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:39:48.942203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:39:48.942203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:39:48.942203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:39:48.942203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:39:48.942203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:39:48.942203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 21:39:48.942203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:48.942203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 554/1098] 10.222.0.35 () {56 vars in 1093 bytes} [Fri May 13 21:39:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:48.946891 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0de5e1-088a-4a89-bdb0-a4a3be144e44 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:49.022011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ff9bf-e2a7-4717-b309-c44355e16ae0 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:49.038594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0de5e1-088a-4a89-bdb0-a4a3be144e44 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:49.053364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a1649-d59a-4ef5-8bc1-f2e33d800dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:49.054009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a1649-d59a-4ef5-8bc1-f2e33d800dea None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:49.054348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a1649-d59a-4ef5-8bc1-f2e33d800dea None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:49.054631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a1649-d59a-4ef5-8bc1-f2e33d800dea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:49.055484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a1649-d59a-4ef5-8bc1-f2e33d800dea None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=b54c31fc82b24b52be69f22d41907044, user.id=60b9b7d7631741439900b296725ce77f) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:49.055796 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a1649-d59a-4ef5-8bc1-f2e33d800dea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:49.057270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a1649-d59a-4ef5-8bc1-f2e33d800dea None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:49.063427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 545/1099] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:39:48 2022] GET /identity/v3/role_assignments?user.id=60b9b7d7631741439900b296725ce77f&scope.project.id=b54c31fc82b24b52be69f22d41907044&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:49.071124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29004c0-050f-447b-a007-dc4d745e0580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:49.071813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29004c0-050f-447b-a007-dc4d745e0580 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:49.072096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29004c0-050f-447b-a007-dc4d745e0580 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:49.072373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29004c0-050f-447b-a007-dc4d745e0580 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:49.073225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29004c0-050f-447b-a007-dc4d745e0580 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:49.073536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29004c0-050f-447b-a007-dc4d745e0580 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:49.075668 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29004c0-050f-447b-a007-dc4d745e0580 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:49.095011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 555/1100] 10.222.0.35 () {56 vars in 1133 bytes} [Fri May 13 21:39:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:49.100824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a1649-d59a-4ef5-8bc1-f2e33d800dea None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:49.203881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a1649-d59a-4ef5-8bc1-f2e33d800dea None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:49.238087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87903e5c-3330-4818-ae0a-6b22413003a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:49.238853 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87903e5c-3330-4818-ae0a-6b22413003a7 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:49.239329 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87903e5c-3330-4818-ae0a-6b22413003a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:49.239794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87903e5c-3330-4818-ae0a-6b22413003a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:49.306016 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87903e5c-3330-4818-ae0a-6b22413003a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:49.307417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87903e5c-3330-4818-ae0a-6b22413003a7 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:49.398673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 546/1101] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:39:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:39:49.404620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29004c0-050f-447b-a007-dc4d745e0580 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:49.498356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29004c0-050f-447b-a007-dc4d745e0580 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:49.531097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681e9f3-2745-4af5-8f20-4303961c720f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:49.531878 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681e9f3-2745-4af5-8f20-4303961c720f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:49.532161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681e9f3-2745-4af5-8f20-4303961c720f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:49.532435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681e9f3-2745-4af5-8f20-4303961c720f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:49.541936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681e9f3-2745-4af5-8f20-4303961c720f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:49.543446 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681e9f3-2745-4af5-8f20-4303961c720f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-9681e9f3-2745-4af5-8f20-4303961c720f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:39:49.562114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:49.563258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:49.564149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:39:49.564149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:39:49.564149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:39:49.564149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:49.564149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 556/1102] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:39:49.569044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87903e5c-3330-4818-ae0a-6b22413003a7 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:49.662310 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87903e5c-3330-4818-ae0a-6b22413003a7 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:49.694505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5761365-5860-4253-a7d0-a784c8b607cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:49.695187 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5761365-5860-4253-a7d0-a784c8b607cf None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:49.695472 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5761365-5860-4253-a7d0-a784c8b607cf None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:49.695751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5761365-5860-4253-a7d0-a784c8b607cf None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:49.696464 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5761365-5860-4253-a7d0-a784c8b607cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:49.696770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5761365-5860-4253-a7d0-a784c8b607cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:49.698151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5761365-5860-4253-a7d0-a784c8b607cf None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:49.709692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 547/1103] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:49.715756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681e9f3-2745-4af5-8f20-4303961c720f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:49.831331 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681e9f3-2745-4af5-8f20-4303961c720f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:49.863853 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80961f-f324-47dc-ab24-3932ce4554ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:49.864632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80961f-f324-47dc-ab24-3932ce4554ec None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:49.864914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80961f-f324-47dc-ab24-3932ce4554ec None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:49.865189 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80961f-f324-47dc-ab24-3932ce4554ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:49.871658 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80961f-f324-47dc-ab24-3932ce4554ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:49.873100 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80961f-f324-47dc-ab24-3932ce4554ec None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-6a80961f-f324-47dc-ab24-3932ce4554ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:39:49.886575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:39:49.887729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:39:49.889036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:39:49.889036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:39:49.889036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:39:49.889036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:39:49.889036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:39:49.889036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:39:49.889036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:39:49.889036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:39:49.889036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 557/1104] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:39:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:39:49.893587 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5761365-5860-4253-a7d0-a784c8b607cf None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:49.988006 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5761365-5860-4253-a7d0-a784c8b607cf None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:50.019721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d81e8-40ea-42f6-810c-c37c320815d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:50.020365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d81e8-40ea-42f6-810c-c37c320815d9 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:50.020642 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d81e8-40ea-42f6-810c-c37c320815d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:50.020911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d81e8-40ea-42f6-810c-c37c320815d9 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:50.021677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d81e8-40ea-42f6-810c-c37c320815d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:50.021979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d81e8-40ea-42f6-810c-c37c320815d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:50.023430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d81e8-40ea-42f6-810c-c37c320815d9 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:50.034861 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 548/1105] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:39:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:50.040860 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80961f-f324-47dc-ab24-3932ce4554ec None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:50.140386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80961f-f324-47dc-ab24-3932ce4554ec None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:50.171805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb59a3b-d671-4cf7-9e9b-611040517412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:50.172446 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb59a3b-d671-4cf7-9e9b-611040517412 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:50.172730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb59a3b-d671-4cf7-9e9b-611040517412 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:50.173001 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb59a3b-d671-4cf7-9e9b-611040517412 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:50.173805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb59a3b-d671-4cf7-9e9b-611040517412 None admin] RBAC: Adding query filter params (role.id=708710710e3e44dca3a6d11cbc2b6dd6, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=316ef43c6c304bd3a27e085bf4da8a5e) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:50.174110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb59a3b-d671-4cf7-9e9b-611040517412 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:50.175553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb59a3b-d671-4cf7-9e9b-611040517412 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:50.181098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 558/1106] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:39:50 2022] GET /identity/v3/role_assignments?user.id=316ef43c6c304bd3a27e085bf4da8a5e&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=708710710e3e44dca3a6d11cbc2b6dd6 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:39:51.307483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd491ff5-ac00-4095-a4d7-f7cf4050faac None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:51.308021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd491ff5-ac00-4095-a4d7-f7cf4050faac None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:51.308021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd491ff5-ac00-4095-a4d7-f7cf4050faac None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:51.309593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 549/1107] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:51.314918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286f4df-a475-4b35-b6bd-ee439ce1d64e None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:51.315328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286f4df-a475-4b35-b6bd-ee439ce1d64e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:51.315477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286f4df-a475-4b35-b6bd-ee439ce1d64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:51.331623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-f286f4df-a475-4b35-b6bd-ee439ce1d64e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:51.332803 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f286f4df-a475-4b35-b6bd-ee439ce1d64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:51.394984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 559/1108] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:51.420387 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd491ff5-ac00-4095-a4d7-f7cf4050faac None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:51.514974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd491ff5-ac00-4095-a4d7-f7cf4050faac None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:51.546325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb2e99-b75b-4e7a-8a90-fa37bed33f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:51.547126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb2e99-b75b-4e7a-8a90-fa37bed33f8c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:51.547408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb2e99-b75b-4e7a-8a90-fa37bed33f8c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:51.547684 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb2e99-b75b-4e7a-8a90-fa37bed33f8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:51.548840 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb2e99-b75b-4e7a-8a90-fa37bed33f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:51.550272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb2e99-b75b-4e7a-8a90-fa37bed33f8c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:51.551893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 550/1109] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:39:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:51.557417 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286f4df-a475-4b35-b6bd-ee439ce1d64e None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:51.648637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286f4df-a475-4b35-b6bd-ee439ce1d64e None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:51.680165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd771f9-ce1b-4c2a-a14a-c0ef2e0e3438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:51.680759 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd771f9-ce1b-4c2a-a14a-c0ef2e0e3438 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:51.681039 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd771f9-ce1b-4c2a-a14a-c0ef2e0e3438 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:51.681316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd771f9-ce1b-4c2a-a14a-c0ef2e0e3438 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:51.682022 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd771f9-ce1b-4c2a-a14a-c0ef2e0e3438 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:51.683488 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd771f9-ce1b-4c2a-a14a-c0ef2e0e3438 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:51.702737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 560/1110] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:39:51 2022] POST /identity/v3/users => generated 286 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:39:53.948225 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be391e4-8ddf-4784-8e29-571b44b70aa9 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:53.948782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be391e4-8ddf-4784-8e29-571b44b70aa9 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:53.948782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be391e4-8ddf-4784-8e29-571b44b70aa9 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:53.950652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 551/1111] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:53.955858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904082c-4b86-4581-b980-0038c93f27f1 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:53.956290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904082c-4b86-4581-b980-0038c93f27f1 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:53.956419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904082c-4b86-4581-b980-0038c93f27f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:53.973210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7904082c-4b86-4581-b980-0038c93f27f1 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:53.974391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7904082c-4b86-4581-b980-0038c93f27f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:54.037008 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 561/1112] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:54.060356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be391e4-8ddf-4784-8e29-571b44b70aa9 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:54.154925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be391e4-8ddf-4784-8e29-571b44b70aa9 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:54.186568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024a425-b72f-49c2-b3e1-fdcefb182feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:54.187435 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024a425-b72f-49c2-b3e1-fdcefb182feb None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:54.187714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024a425-b72f-49c2-b3e1-fdcefb182feb None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:54.187988 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024a425-b72f-49c2-b3e1-fdcefb182feb None admin] PATH_INFO: `/v3/roles/a29f4364896c46dd84dcee5b969b9898` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:54.189475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024a425-b72f-49c2-b3e1-fdcefb182feb None admin] RBAC: Authorizing `identity:get_role(role_id=a29f4364896c46dd84dcee5b969b9898)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:54.190878 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024a425-b72f-49c2-b3e1-fdcefb182feb None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:54.192250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 552/1113] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:39:54 2022] GET /identity/v3/roles/a29f4364896c46dd84dcee5b969b9898 => generated 223 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:54.197952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904082c-4b86-4581-b980-0038c93f27f1 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:54.289308 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904082c-4b86-4581-b980-0038c93f27f1 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:54.321102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d92160-1291-4627-bbd7-f75720c24c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:54.321950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d92160-1291-4627-bbd7-f75720c24c31 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:54.322229 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d92160-1291-4627-bbd7-f75720c24c31 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:54.322503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d92160-1291-4627-bbd7-f75720c24c31 None admin] PATH_INFO: `/v3/users/6abdb43d74b54f6ca2c9e682f001e569` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:54.335385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d92160-1291-4627-bbd7-f75720c24c31 None admin] RBAC: Authorizing `identity:get_user(user_id=6abdb43d74b54f6ca2c9e682f001e569)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:54.336818 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d92160-1291-4627-bbd7-f75720c24c31 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:54.338410 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 562/1114] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:39:54 2022] GET /identity/v3/users/6abdb43d74b54f6ca2c9e682f001e569 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:54.343942 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024a425-b72f-49c2-b3e1-fdcefb182feb None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:54.436879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024a425-b72f-49c2-b3e1-fdcefb182feb None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:54.468838 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f495a-54a2-40dc-8e0d-9aacf38158b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:54.469475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f495a-54a2-40dc-8e0d-9aacf38158b5 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:54.469755 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f495a-54a2-40dc-8e0d-9aacf38158b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:54.470029 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f495a-54a2-40dc-8e0d-9aacf38158b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:54.533408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f495a-54a2-40dc-8e0d-9aacf38158b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:54.534940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f495a-54a2-40dc-8e0d-9aacf38158b5 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:54.629150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 553/1115] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:39:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:39:54.634904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d92160-1291-4627-bbd7-f75720c24c31 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:54.726927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d92160-1291-4627-bbd7-f75720c24c31 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:54.759430 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a61c28-6b5c-406c-bcbe-3e022a2a8c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:54.760257 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a61c28-6b5c-406c-bcbe-3e022a2a8c22 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:54.760536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a61c28-6b5c-406c-bcbe-3e022a2a8c22 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:54.760809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a61c28-6b5c-406c-bcbe-3e022a2a8c22 None admin] PATH_INFO: `/v3/projects/b54c31fc82b24b52be69f22d41907044` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:54.762030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a61c28-6b5c-406c-bcbe-3e022a2a8c22 None admin] RBAC: Authorizing `identity:get_project(project_id=b54c31fc82b24b52be69f22d41907044)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:54.763482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a61c28-6b5c-406c-bcbe-3e022a2a8c22 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:54.765174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 563/1116] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:39:54 2022] GET /identity/v3/projects/b54c31fc82b24b52be69f22d41907044 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:54.771125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f495a-54a2-40dc-8e0d-9aacf38158b5 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:54.864231 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f495a-54a2-40dc-8e0d-9aacf38158b5 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:54.900239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dfa4f-f63d-4b10-bce3-562a00efb917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:54.900813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dfa4f-f63d-4b10-bce3-562a00efb917 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:54.901090 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dfa4f-f63d-4b10-bce3-562a00efb917 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:54.901358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dfa4f-f63d-4b10-bce3-562a00efb917 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:54.902667 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dfa4f-f63d-4b10-bce3-562a00efb917 None admin] RBAC: Adding query filter params (role.id=a29f4364896c46dd84dcee5b969b9898, scope.project.id=b54c31fc82b24b52be69f22d41907044, user.id=6abdb43d74b54f6ca2c9e682f001e569) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:39:54.902999 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dfa4f-f63d-4b10-bce3-562a00efb917 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:54.904423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dfa4f-f63d-4b10-bce3-562a00efb917 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:54.909560 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 554/1117] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:39:54 2022] GET /identity/v3/role_assignments?user.id=6abdb43d74b54f6ca2c9e682f001e569&scope.project.id=b54c31fc82b24b52be69f22d41907044&role.id=a29f4364896c46dd84dcee5b969b9898 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:57.087040 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68698358-1ed9-4481-9938-e13f29e857de None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:57.087570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68698358-1ed9-4481-9938-e13f29e857de None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:57.087570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68698358-1ed9-4481-9938-e13f29e857de None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:57.088943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 564/1118] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:57.095100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c248d15-99e0-4e5b-bdf1-929b94d755b3 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:57.095483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c248d15-99e0-4e5b-bdf1-929b94d755b3 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:57.095649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c248d15-99e0-4e5b-bdf1-929b94d755b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:57.121954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-4c248d15-99e0-4e5b-bdf1-929b94d755b3 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:57.123171 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4c248d15-99e0-4e5b-bdf1-929b94d755b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:39:57.186174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 555/1119] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:39:57.208859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68698358-1ed9-4481-9938-e13f29e857de None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:57.302687 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68698358-1ed9-4481-9938-e13f29e857de None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:57.334561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b7d4c-3d10-47ed-89b5-43f03953fed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:57.335418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b7d4c-3d10-47ed-89b5-43f03953fed5 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:57.335700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b7d4c-3d10-47ed-89b5-43f03953fed5 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:57.335977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b7d4c-3d10-47ed-89b5-43f03953fed5 None admin] PATH_INFO: `/v3/roles/a29f4364896c46dd84dcee5b969b9898` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:57.337470 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b7d4c-3d10-47ed-89b5-43f03953fed5 None admin] RBAC: Authorizing `identity:get_role(role_id=a29f4364896c46dd84dcee5b969b9898)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:57.338810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b7d4c-3d10-47ed-89b5-43f03953fed5 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:57.340166 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 565/1120] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:39:57 2022] GET /identity/v3/roles/a29f4364896c46dd84dcee5b969b9898 => generated 223 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:57.345751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c248d15-99e0-4e5b-bdf1-929b94d755b3 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:57.438884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c248d15-99e0-4e5b-bdf1-929b94d755b3 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:57.471258 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e08ce8-404b-4973-9895-16e616ac6b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:57.472109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e08ce8-404b-4973-9895-16e616ac6b4b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:57.472393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e08ce8-404b-4973-9895-16e616ac6b4b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:57.472672 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e08ce8-404b-4973-9895-16e616ac6b4b None admin] PATH_INFO: `/v3/users/6abdb43d74b54f6ca2c9e682f001e569` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:57.473848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e08ce8-404b-4973-9895-16e616ac6b4b None admin] RBAC: Authorizing `identity:get_user(user_id=6abdb43d74b54f6ca2c9e682f001e569)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:57.475313 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e08ce8-404b-4973-9895-16e616ac6b4b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:57.476939 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 556/1121] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:39:57 2022] GET /identity/v3/users/6abdb43d74b54f6ca2c9e682f001e569 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:57.483350 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b7d4c-3d10-47ed-89b5-43f03953fed5 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:57.575986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b7d4c-3d10-47ed-89b5-43f03953fed5 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:57.607411 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a8c43-a03e-4bcc-8c74-4306d2fc5d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:57.608186 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a8c43-a03e-4bcc-8c74-4306d2fc5d52 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:57.608468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a8c43-a03e-4bcc-8c74-4306d2fc5d52 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:57.608745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a8c43-a03e-4bcc-8c74-4306d2fc5d52 None admin] PATH_INFO: `/v3/projects/b54c31fc82b24b52be69f22d41907044` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:57.609901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a8c43-a03e-4bcc-8c74-4306d2fc5d52 None admin] RBAC: Authorizing `identity:get_project(project_id=b54c31fc82b24b52be69f22d41907044)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:57.611368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a8c43-a03e-4bcc-8c74-4306d2fc5d52 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:57.613041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 566/1122] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:39:57 2022] GET /identity/v3/projects/b54c31fc82b24b52be69f22d41907044 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:39:57.618813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e08ce8-404b-4973-9895-16e616ac6b4b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:39:57.728052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e08ce8-404b-4973-9895-16e616ac6b4b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:39:57.759505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de164ac6-f107-4da6-aa39-4b4aaf03ff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:39:57.760374 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de164ac6-f107-4da6-aa39-4b4aaf03ff70 None admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:57.760653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de164ac6-f107-4da6-aa39-4b4aaf03ff70 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:57.760925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de164ac6-f107-4da6-aa39-4b4aaf03ff70 None admin] PATH_INFO: `/v3/projects/b54c31fc82b24b52be69f22d41907044/users/6abdb43d74b54f6ca2c9e682f001e569/roles/a29f4364896c46dd84dcee5b969b9898` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:57.763669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de164ac6-f107-4da6-aa39-4b4aaf03ff70 None admin] RBAC: Authorizing `identity:create_grant(project_id=b54c31fc82b24b52be69f22d41907044, user_id=6abdb43d74b54f6ca2c9e682f001e569, role_id=a29f4364896c46dd84dcee5b969b9898)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:39:57.765096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de164ac6-f107-4da6-aa39-4b4aaf03ff70 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:39:57.777819 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 557/1123] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:39:57 2022] PUT /identity/v3/projects/b54c31fc82b24b52be69f22d41907044/users/6abdb43d74b54f6ca2c9e682f001e569/roles/a29f4364896c46dd84dcee5b969b9898 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:39:59.963438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a306aa-6ef9-43d2-a949-77c7eb17bb09 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:59.963990 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a306aa-6ef9-43d2-a949-77c7eb17bb09 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:59.963990 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a306aa-6ef9-43d2-a949-77c7eb17bb09 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:59.965986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 567/1124] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:39:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:39:59.971278 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c8c45-793b-47ac-98e7-6928864aa781 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:39:59.971681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c8c45-793b-47ac-98e7-6928864aa781 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:39:59.971841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c8c45-793b-47ac-98e7-6928864aa781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:39:59.987928 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ab1c8c45-793b-47ac-98e7-6928864aa781 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:39:59.989098 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ab1c8c45-793b-47ac-98e7-6928864aa781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:40:00.052214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 558/1125] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:39:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:40:00.076206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a306aa-6ef9-43d2-a949-77c7eb17bb09 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:00.171288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a306aa-6ef9-43d2-a949-77c7eb17bb09 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:00.202292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28501d4f-0e69-48e5-a01c-452da7fbdc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:00.203162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28501d4f-0e69-48e5-a01c-452da7fbdc7a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:00.203444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28501d4f-0e69-48e5-a01c-452da7fbdc7a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:00.203718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28501d4f-0e69-48e5-a01c-452da7fbdc7a None admin] PATH_INFO: `/v3/roles/a29f4364896c46dd84dcee5b969b9898` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:00.205285 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28501d4f-0e69-48e5-a01c-452da7fbdc7a None admin] RBAC: Authorizing `identity:get_role(role_id=a29f4364896c46dd84dcee5b969b9898)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:00.206656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28501d4f-0e69-48e5-a01c-452da7fbdc7a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:00.208045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 568/1126] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:40:00 2022] GET /identity/v3/roles/a29f4364896c46dd84dcee5b969b9898 => generated 223 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:00.213694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c8c45-793b-47ac-98e7-6928864aa781 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:00.305486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c8c45-793b-47ac-98e7-6928864aa781 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:00.336110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a80ff-40b8-483b-8d23-609ceb41564f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:00.336945 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a80ff-40b8-483b-8d23-609ceb41564f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:00.337229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a80ff-40b8-483b-8d23-609ceb41564f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:00.337498 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a80ff-40b8-483b-8d23-609ceb41564f None admin] PATH_INFO: `/v3/users/6abdb43d74b54f6ca2c9e682f001e569` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:00.338658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11a80ff-40b8-483b-8d23-609ceb41564f None admin] RBAC: Authorizing `identity:get_user(user_id=6abdb43d74b54f6ca2c9e682f001e569)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:00.340612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11a80ff-40b8-483b-8d23-609ceb41564f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:00.342206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 559/1127] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:40:00 2022] GET /identity/v3/users/6abdb43d74b54f6ca2c9e682f001e569 => generated 286 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:00.347633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28501d4f-0e69-48e5-a01c-452da7fbdc7a None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:00.440580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28501d4f-0e69-48e5-a01c-452da7fbdc7a None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:00.473298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d84cc-5985-40b8-8ffa-f08a0b12db42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:00.473865 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d84cc-5985-40b8-8ffa-f08a0b12db42 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:00.474147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d84cc-5985-40b8-8ffa-f08a0b12db42 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:00.474424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d84cc-5985-40b8-8ffa-f08a0b12db42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:00.537462 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d84cc-5985-40b8-8ffa-f08a0b12db42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:00.538930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d84cc-5985-40b8-8ffa-f08a0b12db42 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:00.630164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 569/1128] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:40:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:40:00.636147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a80ff-40b8-483b-8d23-609ceb41564f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:00.728095 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a80ff-40b8-483b-8d23-609ceb41564f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:00.759576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f34f4-4740-469c-9209-6dbfb40a5c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:00.760407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f34f4-4740-469c-9209-6dbfb40a5c36 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:00.760684 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f34f4-4740-469c-9209-6dbfb40a5c36 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:00.760956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f34f4-4740-469c-9209-6dbfb40a5c36 None admin] PATH_INFO: `/v3/projects/b54c31fc82b24b52be69f22d41907044` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:00.762673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f34f4-4740-469c-9209-6dbfb40a5c36 None admin] RBAC: Authorizing `identity:get_project(project_id=b54c31fc82b24b52be69f22d41907044)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:00.764128 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f34f4-4740-469c-9209-6dbfb40a5c36 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:00.765784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 560/1129] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:40:00 2022] GET /identity/v3/projects/b54c31fc82b24b52be69f22d41907044 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:40:00.771720 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d84cc-5985-40b8-8ffa-f08a0b12db42 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:00.862892 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d84cc-5985-40b8-8ffa-f08a0b12db42 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:00.898392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca7ec8-9d69-453f-afda-da07193bf8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:00.899059 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca7ec8-9d69-453f-afda-da07193bf8b3 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:00.899346 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca7ec8-9d69-453f-afda-da07193bf8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:00.899809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca7ec8-9d69-453f-afda-da07193bf8b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:00.900503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca7ec8-9d69-453f-afda-da07193bf8b3 None admin] RBAC: Adding query filter params (role.id=a29f4364896c46dd84dcee5b969b9898, scope.project.id=b54c31fc82b24b52be69f22d41907044, user.id=6abdb43d74b54f6ca2c9e682f001e569) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:40:00.900815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca7ec8-9d69-453f-afda-da07193bf8b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:00.902411 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca7ec8-9d69-453f-afda-da07193bf8b3 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:00.907998 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 570/1130] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:40:00 2022] GET /identity/v3/role_assignments?user.id=6abdb43d74b54f6ca2c9e682f001e569&scope.project.id=b54c31fc82b24b52be69f22d41907044&role.id=a29f4364896c46dd84dcee5b969b9898 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:03.090772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc217fe5-f292-4033-b3b8-f1462b009675 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:03.091304 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc217fe5-f292-4033-b3b8-f1462b009675 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:03.091304 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc217fe5-f292-4033-b3b8-f1462b009675 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:03.092676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 561/1131] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:40:03.097731 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b0d70-1021-43eb-98c6-fd3f7fb3ab5e None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:03.098167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b0d70-1021-43eb-98c6-fd3f7fb3ab5e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:03.098286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b0d70-1021-43eb-98c6-fd3f7fb3ab5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:03.114432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-645b0d70-1021-43eb-98c6-fd3f7fb3ab5e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:40:03.115632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-645b0d70-1021-43eb-98c6-fd3f7fb3ab5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:40:03.177955 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 571/1132] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:40:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:40:03.201493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc217fe5-f292-4033-b3b8-f1462b009675 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:03.295196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc217fe5-f292-4033-b3b8-f1462b009675 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:03.327174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245d626-d368-466d-b764-c3553936b6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:03.327924 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245d626-d368-466d-b764-c3553936b6d0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:03.328201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245d626-d368-466d-b764-c3553936b6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:03.328473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245d626-d368-466d-b764-c3553936b6d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:03.329617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245d626-d368-466d-b764-c3553936b6d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:03.331063 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245d626-d368-466d-b764-c3553936b6d0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:03.332629 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 562/1133] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:40:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:03.338193 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b0d70-1021-43eb-98c6-fd3f7fb3ab5e None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:03.430342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b0d70-1021-43eb-98c6-fd3f7fb3ab5e None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:03.462517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b326f9-a5d9-4320-9519-b4bc024476e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:03.463195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b326f9-a5d9-4320-9519-b4bc024476e9 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:03.463477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b326f9-a5d9-4320-9519-b4bc024476e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:03.463751 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b326f9-a5d9-4320-9519-b4bc024476e9 None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:03.464471 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b326f9-a5d9-4320-9519-b4bc024476e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:03.465892 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b326f9-a5d9-4320-9519-b4bc024476e9 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:03.490161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 572/1134] 10.222.0.35 () {60 vars in 1141 bytes} [Fri May 13 21:40:03 2022] POST /identity/v3/projects => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:40:05.733201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3707fe-9912-4363-a730-7bf04b1eafb6 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:05.733792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3707fe-9912-4363-a730-7bf04b1eafb6 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:05.733792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3707fe-9912-4363-a730-7bf04b1eafb6 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:05.735164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 563/1135] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:40:05.740376 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db74f4-e382-4462-a0d7-b0579ff12ebd None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:05.740792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db74f4-e382-4462-a0d7-b0579ff12ebd None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:05.740938 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db74f4-e382-4462-a0d7-b0579ff12ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:05.757382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d1db74f4-e382-4462-a0d7-b0579ff12ebd None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:40:05.758554 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d1db74f4-e382-4462-a0d7-b0579ff12ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:40:05.820489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 573/1136] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:40:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:40:05.845634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3707fe-9912-4363-a730-7bf04b1eafb6 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:05.942860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3707fe-9912-4363-a730-7bf04b1eafb6 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:05.978608 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faed11-69c6-42c2-96d3-e7406e2396e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:05.979404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faed11-69c6-42c2-96d3-e7406e2396e0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:05.979683 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faed11-69c6-42c2-96d3-e7406e2396e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:05.979982 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faed11-69c6-42c2-96d3-e7406e2396e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:05.981130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3faed11-69c6-42c2-96d3-e7406e2396e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:05.982551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3faed11-69c6-42c2-96d3-e7406e2396e0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:05.984244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 564/1137] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:40:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:05.989979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db74f4-e382-4462-a0d7-b0579ff12ebd None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:06.082466 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db74f4-e382-4462-a0d7-b0579ff12ebd None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:06.114011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6c2b3-46f7-4dea-bec0-b38adf58421f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:06.114662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6c2b3-46f7-4dea-bec0-b38adf58421f None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:06.114970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6c2b3-46f7-4dea-bec0-b38adf58421f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:06.115241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6c2b3-46f7-4dea-bec0-b38adf58421f None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:06.115944 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6c2b3-46f7-4dea-bec0-b38adf58421f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:06.117370 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6c2b3-46f7-4dea-bec0-b38adf58421f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:06.137108 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 574/1138] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:40:05 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:40:08.412912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2bbd0-4a1d-4a6a-82b1-cc4153264fa7 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:08.413416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2bbd0-4a1d-4a6a-82b1-cc4153264fa7 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:08.413416 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2bbd0-4a1d-4a6a-82b1-cc4153264fa7 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:08.414762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 565/1139] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:40:08.420046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd7d33-bb24-416d-96f6-36f4ad61d5e0 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:08.420542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd7d33-bb24-416d-96f6-36f4ad61d5e0 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:08.420650 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd7d33-bb24-416d-96f6-36f4ad61d5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:08.437151 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-6afd7d33-bb24-416d-96f6-36f4ad61d5e0 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:40:08.438320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-6afd7d33-bb24-416d-96f6-36f4ad61d5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:40:08.514277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 575/1140] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:40:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:40:08.537029 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2bbd0-4a1d-4a6a-82b1-cc4153264fa7 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:08.630739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2bbd0-4a1d-4a6a-82b1-cc4153264fa7 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:08.661787 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538d1e5-6367-4828-af01-2669d85b6d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:08.662553 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538d1e5-6367-4828-af01-2669d85b6d2e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:08.662862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538d1e5-6367-4828-af01-2669d85b6d2e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:08.663185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538d1e5-6367-4828-af01-2669d85b6d2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:08.672558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538d1e5-6367-4828-af01-2669d85b6d2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-c538d1e5-6367-4828-af01-2669d85b6d2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:40:08.679949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:40:08.681067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:40:08.682030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:40:08.682030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:40:08.682030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:40:08.682030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:40:08.682030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:40:08.682030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:40:08.682030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:40:08.682030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:40:08.682030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:40:08.682030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:40:08.682030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:40:08.682030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:40:08.682030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 566/1141] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:40:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:40:08.687325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd7d33-bb24-416d-96f6-36f4ad61d5e0 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:08.779626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd7d33-bb24-416d-96f6-36f4ad61d5e0 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:08.812054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74878bd0-a055-4a98-bb71-22ba90fa7ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:08.812705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74878bd0-a055-4a98-bb71-22ba90fa7ed5 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:08.812987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74878bd0-a055-4a98-bb71-22ba90fa7ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:08.813261 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74878bd0-a055-4a98-bb71-22ba90fa7ed5 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:08.813988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74878bd0-a055-4a98-bb71-22ba90fa7ed5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:40:08.814292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74878bd0-a055-4a98-bb71-22ba90fa7ed5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:08.815693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74878bd0-a055-4a98-bb71-22ba90fa7ed5 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:08.823302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 576/1142] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:40:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:40:08.828951 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538d1e5-6367-4828-af01-2669d85b6d2e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:08.921587 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538d1e5-6367-4828-af01-2669d85b6d2e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:08.953565 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444148dc-bec3-48bd-93f2-1adc2f42b333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:08.954400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444148dc-bec3-48bd-93f2-1adc2f42b333 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:08.954681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444148dc-bec3-48bd-93f2-1adc2f42b333 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:08.954985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444148dc-bec3-48bd-93f2-1adc2f42b333 None admin] PATH_INFO: `/v3/users/58d18226b9974ddeae91915c21cc71d6` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:08.967484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444148dc-bec3-48bd-93f2-1adc2f42b333 None admin] RBAC: Authorizing `identity:get_user(user_id=58d18226b9974ddeae91915c21cc71d6)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:08.969082 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444148dc-bec3-48bd-93f2-1adc2f42b333 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:08.970739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 567/1143] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:40:08 2022] GET /identity/v3/users/58d18226b9974ddeae91915c21cc71d6 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:08.976269 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74878bd0-a055-4a98-bb71-22ba90fa7ed5 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:09.073357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74878bd0-a055-4a98-bb71-22ba90fa7ed5 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:09.104849 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e127a0-bc98-4fb2-b59e-418523326b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:09.105965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e127a0-bc98-4fb2-b59e-418523326b34 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:09.106247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e127a0-bc98-4fb2-b59e-418523326b34 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:09.106525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e127a0-bc98-4fb2-b59e-418523326b34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:09.168961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e127a0-bc98-4fb2-b59e-418523326b34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:09.170411 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e127a0-bc98-4fb2-b59e-418523326b34 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:09.260687 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 577/1144] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:40:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:40:09.266401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444148dc-bec3-48bd-93f2-1adc2f42b333 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:09.359928 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444148dc-bec3-48bd-93f2-1adc2f42b333 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:09.391346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fde70-c437-4f6d-a5b6-33a7a55b1540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:09.392181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fde70-c437-4f6d-a5b6-33a7a55b1540 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:09.392468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fde70-c437-4f6d-a5b6-33a7a55b1540 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:09.392745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fde70-c437-4f6d-a5b6-33a7a55b1540 None admin] PATH_INFO: `/v3/projects/4043c04f604b4adeaeffed71a1115a02` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:09.393905 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fde70-c437-4f6d-a5b6-33a7a55b1540 None admin] RBAC: Authorizing `identity:get_project(project_id=4043c04f604b4adeaeffed71a1115a02)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:09.395367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fde70-c437-4f6d-a5b6-33a7a55b1540 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:09.397031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 568/1145] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:40:09 2022] GET /identity/v3/projects/4043c04f604b4adeaeffed71a1115a02 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:09.403323 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e127a0-bc98-4fb2-b59e-418523326b34 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:09.494716 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e127a0-bc98-4fb2-b59e-418523326b34 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:09.526243 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649d9af-f5b6-4e7b-92da-235c95e19f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:09.527408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649d9af-f5b6-4e7b-92da-235c95e19f6f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:09.527688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649d9af-f5b6-4e7b-92da-235c95e19f6f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:09.527960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649d9af-f5b6-4e7b-92da-235c95e19f6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:09.528769 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649d9af-f5b6-4e7b-92da-235c95e19f6f None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=4043c04f604b4adeaeffed71a1115a02, user.id=58d18226b9974ddeae91915c21cc71d6) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:40:09.529076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649d9af-f5b6-4e7b-92da-235c95e19f6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:09.530487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649d9af-f5b6-4e7b-92da-235c95e19f6f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:09.535656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 578/1146] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:40:09 2022] GET /identity/v3/role_assignments?user.id=58d18226b9974ddeae91915c21cc71d6&scope.project.id=4043c04f604b4adeaeffed71a1115a02&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 259 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:11.707310 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18402e0b-2d8d-4579-b0c8-92d815fe9444 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:11.707841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18402e0b-2d8d-4579-b0c8-92d815fe9444 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:11.707841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18402e0b-2d8d-4579-b0c8-92d815fe9444 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:11.709228 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 569/1147] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:40:11.714873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ea0ee-9a7f-4761-b7ec-c02f4d8c83f8 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:11.715306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ea0ee-9a7f-4761-b7ec-c02f4d8c83f8 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:11.715426 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ea0ee-9a7f-4761-b7ec-c02f4d8c83f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:11.732075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-1b1ea0ee-9a7f-4761-b7ec-c02f4d8c83f8 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:40:11.733238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-1b1ea0ee-9a7f-4761-b7ec-c02f4d8c83f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:40:11.794808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 579/1148] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:40:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:40:11.816616 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18402e0b-2d8d-4579-b0c8-92d815fe9444 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:11.913601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18402e0b-2d8d-4579-b0c8-92d815fe9444 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:11.945012 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad02cb6-6c02-44b4-a253-1aa844b95417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:11.945776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad02cb6-6c02-44b4-a253-1aa844b95417 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:11.946057 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad02cb6-6c02-44b4-a253-1aa844b95417 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:11.946332 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad02cb6-6c02-44b4-a253-1aa844b95417 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:11.956490 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad02cb6-6c02-44b4-a253-1aa844b95417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-4ad02cb6-6c02-44b4-a253-1aa844b95417 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:40:11.964588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:40:11.965679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:40:11.966562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:40:11.966562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:40:11.966562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:40:11.966562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:40:11.966562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:40:11.966562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:40:11.966562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:40:11.966562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:40:11.966562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:40:11.966562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:40:11.966562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:40:11.966562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:40:11.966562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 570/1149] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:40:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:40:11.971514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ea0ee-9a7f-4761-b7ec-c02f4d8c83f8 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:12.065855 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ea0ee-9a7f-4761-b7ec-c02f4d8c83f8 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:12.097102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f034ac0-f09b-47e1-acbd-ec6a7ea6d8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:12.097744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f034ac0-f09b-47e1-acbd-ec6a7ea6d8f4 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:12.098023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f034ac0-f09b-47e1-acbd-ec6a7ea6d8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:12.098293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f034ac0-f09b-47e1-acbd-ec6a7ea6d8f4 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:12.099047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f034ac0-f09b-47e1-acbd-ec6a7ea6d8f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:40:12.099349 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f034ac0-f09b-47e1-acbd-ec6a7ea6d8f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:12.100704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f034ac0-f09b-47e1-acbd-ec6a7ea6d8f4 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:12.108241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 580/1150] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:40:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:12.113980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad02cb6-6c02-44b4-a253-1aa844b95417 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:12.216020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad02cb6-6c02-44b4-a253-1aa844b95417 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:12.247907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c092b-6982-4aec-bb32-96d0e3e1269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:12.248757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c092b-6982-4aec-bb32-96d0e3e1269e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:12.249038 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c092b-6982-4aec-bb32-96d0e3e1269e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:12.249313 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c092b-6982-4aec-bb32-96d0e3e1269e None admin] PATH_INFO: `/v3/users/58d18226b9974ddeae91915c21cc71d6` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:12.250488 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c092b-6982-4aec-bb32-96d0e3e1269e None admin] RBAC: Authorizing `identity:get_user(user_id=58d18226b9974ddeae91915c21cc71d6)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:12.252215 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c092b-6982-4aec-bb32-96d0e3e1269e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:12.253846 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 571/1151] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:40:12 2022] GET /identity/v3/users/58d18226b9974ddeae91915c21cc71d6 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:12.259563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f034ac0-f09b-47e1-acbd-ec6a7ea6d8f4 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:12.351641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f034ac0-f09b-47e1-acbd-ec6a7ea6d8f4 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:12.382520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec39980-5331-4624-a14b-4a10b9419b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:12.383368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec39980-5331-4624-a14b-4a10b9419b22 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:12.383647 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec39980-5331-4624-a14b-4a10b9419b22 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:12.383914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec39980-5331-4624-a14b-4a10b9419b22 None admin] PATH_INFO: `/v3/projects/4043c04f604b4adeaeffed71a1115a02` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:12.385057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec39980-5331-4624-a14b-4a10b9419b22 None admin] RBAC: Authorizing `identity:get_project(project_id=4043c04f604b4adeaeffed71a1115a02)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:12.387105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec39980-5331-4624-a14b-4a10b9419b22 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:12.388745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 581/1152] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:40:12 2022] GET /identity/v3/projects/4043c04f604b4adeaeffed71a1115a02 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:40:12.394377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c092b-6982-4aec-bb32-96d0e3e1269e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:12.495373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c092b-6982-4aec-bb32-96d0e3e1269e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:12.527918 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c04262-7206-4db3-bf2a-33be15b8dbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:12.528793 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c04262-7206-4db3-bf2a-33be15b8dbe1 None admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:12.529077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c04262-7206-4db3-bf2a-33be15b8dbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:12.529354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c04262-7206-4db3-bf2a-33be15b8dbe1 None admin] PATH_INFO: `/v3/projects/4043c04f604b4adeaeffed71a1115a02/users/58d18226b9974ddeae91915c21cc71d6/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:12.531332 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c04262-7206-4db3-bf2a-33be15b8dbe1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4043c04f604b4adeaeffed71a1115a02, user_id=58d18226b9974ddeae91915c21cc71d6, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:12.532766 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c04262-7206-4db3-bf2a-33be15b8dbe1 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:12.545447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 572/1153] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:40:12 2022] PUT /identity/v3/projects/4043c04f604b4adeaeffed71a1115a02/users/58d18226b9974ddeae91915c21cc71d6/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:40:14.813565 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860ee70-c376-4d50-8bdd-1bb9f88e8a7c None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:14.814098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860ee70-c376-4d50-8bdd-1bb9f88e8a7c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:14.814098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860ee70-c376-4d50-8bdd-1bb9f88e8a7c None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:14.815558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 582/1154] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:40:14.820897 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7635fb0-1c39-416d-9e54-91ad84db6284 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:14.821316 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7635fb0-1c39-416d-9e54-91ad84db6284 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:14.821457 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7635fb0-1c39-416d-9e54-91ad84db6284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:14.838086 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-e7635fb0-1c39-416d-9e54-91ad84db6284 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:40:14.839934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-e7635fb0-1c39-416d-9e54-91ad84db6284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:40:14.901157 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 573/1155] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:40:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:40:14.924706 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860ee70-c376-4d50-8bdd-1bb9f88e8a7c None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:15.021424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860ee70-c376-4d50-8bdd-1bb9f88e8a7c None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:15.052285 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a5462-3fe8-48c4-87e6-c2e1f3db6800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:15.053047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a5462-3fe8-48c4-87e6-c2e1f3db6800 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:15.053325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a5462-3fe8-48c4-87e6-c2e1f3db6800 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:15.053595 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a5462-3fe8-48c4-87e6-c2e1f3db6800 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:15.064079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a5462-3fe8-48c4-87e6-c2e1f3db6800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-c11a5462-3fe8-48c4-87e6-c2e1f3db6800 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:40:15.071408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:40:15.072509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:40:15.074127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:40:15.074127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:40:15.074127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:40:15.074127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:40:15.074127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:40:15.074127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:40:15.074127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:40:15.074127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:40:15.074127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:40:15.074127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:40:15.074127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:40:15.074127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:40:15.074127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 583/1156] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:40:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:40:15.079098 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7635fb0-1c39-416d-9e54-91ad84db6284 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:15.171317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7635fb0-1c39-416d-9e54-91ad84db6284 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:15.203837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843dd5b-ce16-4064-b5ae-20d2092d8070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:15.204493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843dd5b-ce16-4064-b5ae-20d2092d8070 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:15.204777 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843dd5b-ce16-4064-b5ae-20d2092d8070 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:15.205054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843dd5b-ce16-4064-b5ae-20d2092d8070 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:15.205788 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843dd5b-ce16-4064-b5ae-20d2092d8070 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:40:15.206093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843dd5b-ce16-4064-b5ae-20d2092d8070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:15.207565 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843dd5b-ce16-4064-b5ae-20d2092d8070 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:15.215264 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 574/1157] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:40:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:15.220884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a5462-3fe8-48c4-87e6-c2e1f3db6800 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:15.313106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a5462-3fe8-48c4-87e6-c2e1f3db6800 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:15.344436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce426d-fd2a-4324-bfe1-507afb96e91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:15.345222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce426d-fd2a-4324-bfe1-507afb96e91b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:15.345497 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce426d-fd2a-4324-bfe1-507afb96e91b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:15.345766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce426d-fd2a-4324-bfe1-507afb96e91b None admin] PATH_INFO: `/v3/users/58d18226b9974ddeae91915c21cc71d6` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:15.346949 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce426d-fd2a-4324-bfe1-507afb96e91b None admin] RBAC: Authorizing `identity:get_user(user_id=58d18226b9974ddeae91915c21cc71d6)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:15.348374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce426d-fd2a-4324-bfe1-507afb96e91b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:15.349971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 584/1158] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:40:15 2022] GET /identity/v3/users/58d18226b9974ddeae91915c21cc71d6 => generated 286 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:15.355721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843dd5b-ce16-4064-b5ae-20d2092d8070 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:15.449947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843dd5b-ce16-4064-b5ae-20d2092d8070 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:15.481276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa47442-0b60-4d53-afed-b2e765eca830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:15.481893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa47442-0b60-4d53-afed-b2e765eca830 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:15.482174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa47442-0b60-4d53-afed-b2e765eca830 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:15.482449 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa47442-0b60-4d53-afed-b2e765eca830 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:15.545288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa47442-0b60-4d53-afed-b2e765eca830 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:15.546745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa47442-0b60-4d53-afed-b2e765eca830 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:15.637950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 575/1159] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:40:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:40:15.643735 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce426d-fd2a-4324-bfe1-507afb96e91b None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:15.735599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce426d-fd2a-4324-bfe1-507afb96e91b None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:15.768074 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabad248-f7cb-4734-ac88-e6d64471672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:15.769149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabad248-f7cb-4734-ac88-e6d64471672a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:15.769429 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabad248-f7cb-4734-ac88-e6d64471672a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:15.769698 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabad248-f7cb-4734-ac88-e6d64471672a None admin] PATH_INFO: `/v3/projects/4043c04f604b4adeaeffed71a1115a02` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:15.770869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabad248-f7cb-4734-ac88-e6d64471672a None admin] RBAC: Authorizing `identity:get_project(project_id=4043c04f604b4adeaeffed71a1115a02)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:15.772280 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabad248-f7cb-4734-ac88-e6d64471672a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:15.773915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 585/1160] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:40:15 2022] GET /identity/v3/projects/4043c04f604b4adeaeffed71a1115a02 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:40:15.780240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa47442-0b60-4d53-afed-b2e765eca830 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:15.871897 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa47442-0b60-4d53-afed-b2e765eca830 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:15.903668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635c930-02bc-4421-b5c2-fb1e010089e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:15.904238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635c930-02bc-4421-b5c2-fb1e010089e2 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:15.904517 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635c930-02bc-4421-b5c2-fb1e010089e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:15.904788 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635c930-02bc-4421-b5c2-fb1e010089e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:15.905594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635c930-02bc-4421-b5c2-fb1e010089e2 None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=4043c04f604b4adeaeffed71a1115a02, user.id=58d18226b9974ddeae91915c21cc71d6) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:40:15.905899 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635c930-02bc-4421-b5c2-fb1e010089e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:15.907333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635c930-02bc-4421-b5c2-fb1e010089e2 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:15.912805 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 576/1161] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:40:15 2022] GET /identity/v3/role_assignments?user.id=58d18226b9974ddeae91915c21cc71d6&scope.project.id=4043c04f604b4adeaeffed71a1115a02&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 608 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:18.110534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b240e93-422d-4ea7-a3b6-2c682d326bfd None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:18.111113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b240e93-422d-4ea7-a3b6-2c682d326bfd None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:18.111113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b240e93-422d-4ea7-a3b6-2c682d326bfd None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:18.112494 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 586/1162] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:40:18.117573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f48739-cd9b-44e8-9525-ee4faeb483c9 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:18.117967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f48739-cd9b-44e8-9525-ee4faeb483c9 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:18.118124 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f48739-cd9b-44e8-9525-ee4faeb483c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:18.134468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-43f48739-cd9b-44e8-9525-ee4faeb483c9 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:40:18.135734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-43f48739-cd9b-44e8-9525-ee4faeb483c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:40:18.198501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 577/1163] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:40:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:40:18.220837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b240e93-422d-4ea7-a3b6-2c682d326bfd None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:18.315251 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b240e93-422d-4ea7-a3b6-2c682d326bfd None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:18.346199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe6618-4e61-4d3d-93c1-1db3280b4977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:18.346859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6618-4e61-4d3d-93c1-1db3280b4977 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:18.347139 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6618-4e61-4d3d-93c1-1db3280b4977 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:18.347410 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6618-4e61-4d3d-93c1-1db3280b4977 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:18.408644 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe6618-4e61-4d3d-93c1-1db3280b4977 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:18.410071 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe6618-4e61-4d3d-93c1-1db3280b4977 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:18.500321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 587/1164] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:40:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:40:18.506499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f48739-cd9b-44e8-9525-ee4faeb483c9 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:18.598881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f48739-cd9b-44e8-9525-ee4faeb483c9 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:18.630071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86104ac-9489-4f4e-b232-586ae8ff963d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:18.630887 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86104ac-9489-4f4e-b232-586ae8ff963d None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:18.631170 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86104ac-9489-4f4e-b232-586ae8ff963d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:18.631443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86104ac-9489-4f4e-b232-586ae8ff963d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:18.638329 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86104ac-9489-4f4e-b232-586ae8ff963d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:18.639785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86104ac-9489-4f4e-b232-586ae8ff963d None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-d86104ac-9489-4f4e-b232-586ae8ff963d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:40:18.653967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:40:18.655156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:40:18.656003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 578/1165] 10.222.0.35 () {56 vars in 1117 bytes} [Fri May 13 21:40:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:40:18.661092 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe6618-4e61-4d3d-93c1-1db3280b4977 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:18.751338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe6618-4e61-4d3d-93c1-1db3280b4977 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:18.782972 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff6c0a-6ac5-4e67-891c-cda106c0fc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:18.783604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff6c0a-6ac5-4e67-891c-cda106c0fc38 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:18.783884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff6c0a-6ac5-4e67-891c-cda106c0fc38 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:18.784159 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff6c0a-6ac5-4e67-891c-cda106c0fc38 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:18.784857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff6c0a-6ac5-4e67-891c-cda106c0fc38 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:40:18.785157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff6c0a-6ac5-4e67-891c-cda106c0fc38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:18.786506 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff6c0a-6ac5-4e67-891c-cda106c0fc38 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:18.793227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 588/1166] 10.222.0.35 () {56 vars in 1115 bytes} [Fri May 13 21:40:18 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:20.971177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750be111-982b-470b-8c75-e1af6715210c None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:20.971763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750be111-982b-470b-8c75-e1af6715210c None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:20.971763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750be111-982b-470b-8c75-e1af6715210c None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:20.973097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 579/1167] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:40:20.978340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4e3f4-3c00-4cce-a006-a79cf092c2cb None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:20.978816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4e3f4-3c00-4cce-a006-a79cf092c2cb None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:20.978917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4e3f4-3c00-4cce-a006-a79cf092c2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:20.996467 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-2ec4e3f4-3c00-4cce-a006-a79cf092c2cb None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:40:20.997627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-2ec4e3f4-3c00-4cce-a006-a79cf092c2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:40:21.059452 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 589/1168] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:40:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:40:21.081325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750be111-982b-470b-8c75-e1af6715210c None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:21.180822 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750be111-982b-470b-8c75-e1af6715210c None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:21.212152 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19973d4c-c189-48df-bbbe-ffedb2c722f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:21.212805 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19973d4c-c189-48df-bbbe-ffedb2c722f6 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:21.213088 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19973d4c-c189-48df-bbbe-ffedb2c722f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:21.213371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19973d4c-c189-48df-bbbe-ffedb2c722f6 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:21.214604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19973d4c-c189-48df-bbbe-ffedb2c722f6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:21.215997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19973d4c-c189-48df-bbbe-ffedb2c722f6 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:21.231047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 580/1169] 10.222.0.35 () {60 vars in 1138 bytes} [Fri May 13 21:40:21 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:40:23.437939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc83204-f45d-4c92-a3c4-6be753a044e6 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:23.438478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc83204-f45d-4c92-a3c4-6be753a044e6 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:23.438478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc83204-f45d-4c92-a3c4-6be753a044e6 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:23.439884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 590/1170] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:40:23.445388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453b2a7-a5f7-446f-9979-11a2f19ec7ee None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:23.445784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453b2a7-a5f7-446f-9979-11a2f19ec7ee None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:23.445945 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453b2a7-a5f7-446f-9979-11a2f19ec7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:23.462784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-b453b2a7-a5f7-446f-9979-11a2f19ec7ee None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:40:23.464001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b453b2a7-a5f7-446f-9979-11a2f19ec7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:40:23.526978 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 581/1171] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:40:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:40:23.559012 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc83204-f45d-4c92-a3c4-6be753a044e6 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:23.653981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc83204-f45d-4c92-a3c4-6be753a044e6 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:23.685349 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572781d-7602-4f46-8e93-852ca5ce7685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:23.686186 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572781d-7602-4f46-8e93-852ca5ce7685 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:23.686465 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572781d-7602-4f46-8e93-852ca5ce7685 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:23.686757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572781d-7602-4f46-8e93-852ca5ce7685 None admin] PATH_INFO: `/v3/domains/4ef01a13c76b49b8850f411aaecee17e` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:23.696481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572781d-7602-4f46-8e93-852ca5ce7685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4ef01a13c76b49b8850f411aaecee17e)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:23.697861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572781d-7602-4f46-8e93-852ca5ce7685 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:23.700020 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 591/1172] 10.222.0.35 () {56 vars in 1183 bytes} [Fri May 13 21:40:23 2022] GET /identity/v3/domains/4ef01a13c76b49b8850f411aaecee17e => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:23.705937 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453b2a7-a5f7-446f-9979-11a2f19ec7ee None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:23.797379 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453b2a7-a5f7-446f-9979-11a2f19ec7ee None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:23.828465 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbb8a1-c57b-4064-aaec-d77b40826540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:23.829100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbb8a1-c57b-4064-aaec-d77b40826540 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:23.829378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbb8a1-c57b-4064-aaec-d77b40826540 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:23.829647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbb8a1-c57b-4064-aaec-d77b40826540 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:23.830329 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbb8a1-c57b-4064-aaec-d77b40826540 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:23.831787 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbb8a1-c57b-4064-aaec-d77b40826540 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:23.855394 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 582/1173] 10.222.0.35 () {60 vars in 1141 bytes} [Fri May 13 21:40:23 2022] POST /identity/v3/projects => generated 361 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:40:26.105690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b923d-e051-4241-be8a-a56b1dd132a8 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:26.106228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b923d-e051-4241-be8a-a56b1dd132a8 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:26.106228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b923d-e051-4241-be8a-a56b1dd132a8 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:26.107624 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 592/1174] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:40:26.113005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15652a17-0a90-4c13-bf0c-e6c17f3f090e None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:26.113402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15652a17-0a90-4c13-bf0c-e6c17f3f090e None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:26.113567 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15652a17-0a90-4c13-bf0c-e6c17f3f090e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:26.129983 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-15652a17-0a90-4c13-bf0c-e6c17f3f090e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:40:26.131190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-15652a17-0a90-4c13-bf0c-e6c17f3f090e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:40:26.193881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 583/1175] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:40:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:40:26.219309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b923d-e051-4241-be8a-a56b1dd132a8 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:26.329336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b923d-e051-4241-be8a-a56b1dd132a8 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:26.359843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94112c5-2578-4b2c-b390-83e8bffa3905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:26.360609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94112c5-2578-4b2c-b390-83e8bffa3905 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:26.360885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94112c5-2578-4b2c-b390-83e8bffa3905 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:26.361149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94112c5-2578-4b2c-b390-83e8bffa3905 None admin] PATH_INFO: `/v3/domains/4ef01a13c76b49b8850f411aaecee17e` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:26.362300 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94112c5-2578-4b2c-b390-83e8bffa3905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4ef01a13c76b49b8850f411aaecee17e)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:26.363774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94112c5-2578-4b2c-b390-83e8bffa3905 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:26.365377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 593/1176] 10.222.0.35 () {56 vars in 1183 bytes} [Fri May 13 21:40:26 2022] GET /identity/v3/domains/4ef01a13c76b49b8850f411aaecee17e => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:26.371603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15652a17-0a90-4c13-bf0c-e6c17f3f090e None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:26.463125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15652a17-0a90-4c13-bf0c-e6c17f3f090e None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:26.494100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b6039-d1ca-4d7d-97d7-493f8a49809f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:26.495264 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b6039-d1ca-4d7d-97d7-493f8a49809f None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:26.495548 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b6039-d1ca-4d7d-97d7-493f8a49809f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:26.495829 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b6039-d1ca-4d7d-97d7-493f8a49809f None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:26.496529 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b6039-d1ca-4d7d-97d7-493f8a49809f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:26.497958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b6039-d1ca-4d7d-97d7-493f8a49809f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:26.517593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 584/1177] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:40:26 2022] POST /identity/v3/users => generated 311 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:40:28.745569 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79bebf-f126-4d8f-ad38-823b15e65b07 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:28.746105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79bebf-f126-4d8f-ad38-823b15e65b07 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:28.746105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79bebf-f126-4d8f-ad38-823b15e65b07 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:28.747510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 594/1178] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:40:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:40:28.752979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e05a7-9eff-4eda-a23b-7844cdeda6d2 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:28.753381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e05a7-9eff-4eda-a23b-7844cdeda6d2 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:28.753537 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e05a7-9eff-4eda-a23b-7844cdeda6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:28.770221 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-fa0e05a7-9eff-4eda-a23b-7844cdeda6d2 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:40:28.771426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-fa0e05a7-9eff-4eda-a23b-7844cdeda6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:40:28.834137 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 585/1179] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:40:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:40:28.857216 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79bebf-f126-4d8f-ad38-823b15e65b07 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:28.954571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79bebf-f126-4d8f-ad38-823b15e65b07 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:28.985638 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2f7c2-5398-4895-bd26-2f3d4316febc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:28.986438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2f7c2-5398-4895-bd26-2f3d4316febc None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:28.986738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2f7c2-5398-4895-bd26-2f3d4316febc None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:28.987016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2f7c2-5398-4895-bd26-2f3d4316febc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:28.997610 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2f7c2-5398-4895-bd26-2f3d4316febc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-33f2f7c2-5398-4895-bd26-2f3d4316febc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:40:29.005209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:40:29.006375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:40:29.007525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:40:29.007525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:40:29.007525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:40:29.007525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:40:29.007525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:40:29.007525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:40:29.007525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:40:29.007525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:40:29.007525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:40:29.007525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:40:29.007525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:40:29.007525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:40:29.007525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 595/1180] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:40:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:40:29.012556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e05a7-9eff-4eda-a23b-7844cdeda6d2 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:29.105706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e05a7-9eff-4eda-a23b-7844cdeda6d2 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:29.137809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10052de-38b0-4c9e-b358-e8fa9b215c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:29.138480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10052de-38b0-4c9e-b358-e8fa9b215c83 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:29.138770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10052de-38b0-4c9e-b358-e8fa9b215c83 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:29.139048 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10052de-38b0-4c9e-b358-e8fa9b215c83 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:29.139791 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10052de-38b0-4c9e-b358-e8fa9b215c83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:40:29.140100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10052de-38b0-4c9e-b358-e8fa9b215c83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:29.141491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10052de-38b0-4c9e-b358-e8fa9b215c83 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:29.151710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 586/1181] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:40:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:29.154966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2f7c2-5398-4895-bd26-2f3d4316febc None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:29.256387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2f7c2-5398-4895-bd26-2f3d4316febc None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:29.287314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d6306-b183-4386-b3f3-6f59cdee1f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:29.288154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d6306-b183-4386-b3f3-6f59cdee1f8e None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:29.288433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d6306-b183-4386-b3f3-6f59cdee1f8e None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:29.288706 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d6306-b183-4386-b3f3-6f59cdee1f8e None admin] PATH_INFO: `/v3/users/0d311b95f53e47afae209a2c8f8d4070` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:29.301396 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d6306-b183-4386-b3f3-6f59cdee1f8e None admin] RBAC: Authorizing `identity:get_user(user_id=0d311b95f53e47afae209a2c8f8d4070)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:29.302877 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d6306-b183-4386-b3f3-6f59cdee1f8e None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:29.304514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 596/1182] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:40:29 2022] GET /identity/v3/users/0d311b95f53e47afae209a2c8f8d4070 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:29.310854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10052de-38b0-4c9e-b358-e8fa9b215c83 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:29.404369 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10052de-38b0-4c9e-b358-e8fa9b215c83 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:29.435557 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8364a1f-769e-45d1-ac19-42ea8c4ee42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:29.436185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8364a1f-769e-45d1-ac19-42ea8c4ee42f None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:29.436462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8364a1f-769e-45d1-ac19-42ea8c4ee42f None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:29.436732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8364a1f-769e-45d1-ac19-42ea8c4ee42f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:29.498412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8364a1f-769e-45d1-ac19-42ea8c4ee42f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:29.499885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8364a1f-769e-45d1-ac19-42ea8c4ee42f None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:29.593117 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 587/1183] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:40:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:40:29.598914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d6306-b183-4386-b3f3-6f59cdee1f8e None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:29.691097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d6306-b183-4386-b3f3-6f59cdee1f8e None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:29.722924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6198c-283d-4eb3-85ad-f745f136b8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:29.723752 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6198c-283d-4eb3-85ad-f745f136b8de None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:29.724035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6198c-283d-4eb3-85ad-f745f136b8de None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:29.724312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6198c-283d-4eb3-85ad-f745f136b8de None admin] PATH_INFO: `/v3/projects/d5e41cb798df4b8699eaee70b82b7176` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:29.725463 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6198c-283d-4eb3-85ad-f745f136b8de None admin] RBAC: Authorizing `identity:get_project(project_id=d5e41cb798df4b8699eaee70b82b7176)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:29.726918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6198c-283d-4eb3-85ad-f745f136b8de None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:29.728573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 597/1184] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:40:29 2022] GET /identity/v3/projects/d5e41cb798df4b8699eaee70b82b7176 => generated 361 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:29.734326 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8364a1f-769e-45d1-ac19-42ea8c4ee42f None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:29.827311 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8364a1f-769e-45d1-ac19-42ea8c4ee42f None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:29.859974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb43636-333f-421e-b5ad-5326b2d68c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:29.860547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb43636-333f-421e-b5ad-5326b2d68c0c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:29.860830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb43636-333f-421e-b5ad-5326b2d68c0c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:29.861103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb43636-333f-421e-b5ad-5326b2d68c0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:29.861921 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb43636-333f-421e-b5ad-5326b2d68c0c None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=d5e41cb798df4b8699eaee70b82b7176, user.id=0d311b95f53e47afae209a2c8f8d4070) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:40:29.862230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb43636-333f-421e-b5ad-5326b2d68c0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:29.863674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb43636-333f-421e-b5ad-5326b2d68c0c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:29.868903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 588/1185] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:40:29 2022] GET /identity/v3/role_assignments?user.id=0d311b95f53e47afae209a2c8f8d4070&scope.project.id=d5e41cb798df4b8699eaee70b82b7176&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:32.033741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da2049-9cf7-4ec3-87a5-0729fd04369d None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:32.034272 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da2049-9cf7-4ec3-87a5-0729fd04369d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:32.034272 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da2049-9cf7-4ec3-87a5-0729fd04369d None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:32.035720 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 598/1186] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:40:32.040861 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9292f-3b28-4151-821d-0519fb14b9e1 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:32.041236 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9292f-3b28-4151-821d-0519fb14b9e1 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:32.041452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9292f-3b28-4151-821d-0519fb14b9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:32.057503 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-2bc9292f-3b28-4151-821d-0519fb14b9e1 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:40:32.058728 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2bc9292f-3b28-4151-821d-0519fb14b9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:40:32.121192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 589/1187] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:40:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:40:32.144170 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da2049-9cf7-4ec3-87a5-0729fd04369d None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:32.238930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da2049-9cf7-4ec3-87a5-0729fd04369d None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:32.270661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df79ff-14fb-40ce-97e5-8234aa1a452f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:32.271455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df79ff-14fb-40ce-97e5-8234aa1a452f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:32.271737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df79ff-14fb-40ce-97e5-8234aa1a452f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:32.272009 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df79ff-14fb-40ce-97e5-8234aa1a452f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:32.282017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df79ff-14fb-40ce-97e5-8234aa1a452f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-09df79ff-14fb-40ce-97e5-8234aa1a452f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:40:32.289685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:40:32.290815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:40:32.292385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:40:32.292385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:40:32.292385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:40:32.292385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:40:32.292385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:40:32.292385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:40:32.292385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:40:32.292385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:40:32.292385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:40:32.292385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:40:32.292385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:40:32.292385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:40:32.292385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 599/1188] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:40:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:40:32.297309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9292f-3b28-4151-821d-0519fb14b9e1 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:32.388745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9292f-3b28-4151-821d-0519fb14b9e1 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:32.420534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfbdea-fa46-4684-8a4d-254067923da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:32.421174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfbdea-fa46-4684-8a4d-254067923da2 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:32.421452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfbdea-fa46-4684-8a4d-254067923da2 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:32.421721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfbdea-fa46-4684-8a4d-254067923da2 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:32.422440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfbdea-fa46-4684-8a4d-254067923da2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:40:32.422762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfbdea-fa46-4684-8a4d-254067923da2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:32.424108 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfbdea-fa46-4684-8a4d-254067923da2 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:32.431656 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 590/1189] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:40:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:32.437387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df79ff-14fb-40ce-97e5-8234aa1a452f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:32.531482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df79ff-14fb-40ce-97e5-8234aa1a452f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:32.569762 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77531c-e459-45a5-9a71-0b85add43e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:32.570611 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77531c-e459-45a5-9a71-0b85add43e07 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:32.570913 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77531c-e459-45a5-9a71-0b85add43e07 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:32.571185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77531c-e459-45a5-9a71-0b85add43e07 None admin] PATH_INFO: `/v3/users/0d311b95f53e47afae209a2c8f8d4070` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:32.572367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77531c-e459-45a5-9a71-0b85add43e07 None admin] RBAC: Authorizing `identity:get_user(user_id=0d311b95f53e47afae209a2c8f8d4070)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:32.573823 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77531c-e459-45a5-9a71-0b85add43e07 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:32.575476 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 600/1190] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:40:32 2022] GET /identity/v3/users/0d311b95f53e47afae209a2c8f8d4070 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:32.581132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfbdea-fa46-4684-8a4d-254067923da2 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:32.674641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfbdea-fa46-4684-8a4d-254067923da2 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:32.705631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c3fd2-ac68-4494-969f-49b0db5d1908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:32.706459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c3fd2-ac68-4494-969f-49b0db5d1908 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:32.706762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c3fd2-ac68-4494-969f-49b0db5d1908 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:32.707043 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c3fd2-ac68-4494-969f-49b0db5d1908 None admin] PATH_INFO: `/v3/projects/d5e41cb798df4b8699eaee70b82b7176` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:32.708200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c3fd2-ac68-4494-969f-49b0db5d1908 None admin] RBAC: Authorizing `identity:get_project(project_id=d5e41cb798df4b8699eaee70b82b7176)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:32.709631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c3fd2-ac68-4494-969f-49b0db5d1908 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:32.711317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 591/1191] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:40:32 2022] GET /identity/v3/projects/d5e41cb798df4b8699eaee70b82b7176 => generated 361 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:32.717498 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77531c-e459-45a5-9a71-0b85add43e07 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:32.809223 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77531c-e459-45a5-9a71-0b85add43e07 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:32.840567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07fd5e-fe68-4b19-84dc-1270664a32de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:32.841386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07fd5e-fe68-4b19-84dc-1270664a32de None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:32.841664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07fd5e-fe68-4b19-84dc-1270664a32de None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:32.841933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07fd5e-fe68-4b19-84dc-1270664a32de None admin] PATH_INFO: `/v3/projects/d5e41cb798df4b8699eaee70b82b7176/users/0d311b95f53e47afae209a2c8f8d4070/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:32.843875 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07fd5e-fe68-4b19-84dc-1270664a32de None admin] RBAC: Authorizing `identity:create_grant(project_id=d5e41cb798df4b8699eaee70b82b7176, user_id=0d311b95f53e47afae209a2c8f8d4070, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:32.845369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07fd5e-fe68-4b19-84dc-1270664a32de None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:32.858897 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 601/1192] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:40:32 2022] PUT /identity/v3/projects/d5e41cb798df4b8699eaee70b82b7176/users/0d311b95f53e47afae209a2c8f8d4070/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:40:35.013683 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec1caa-9510-4ffb-ad45-50e054023978 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:35.014229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec1caa-9510-4ffb-ad45-50e054023978 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:35.014229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec1caa-9510-4ffb-ad45-50e054023978 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:35.015643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 592/1193] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:40:35.020708 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a4096-8459-47e8-b434-908f1cb71b57 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:35.021097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a4096-8459-47e8-b434-908f1cb71b57 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:35.021265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a4096-8459-47e8-b434-908f1cb71b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:35.037600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d70a4096-8459-47e8-b434-908f1cb71b57 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:40:35.038783 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d70a4096-8459-47e8-b434-908f1cb71b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:40:35.106096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 602/1194] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:40:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:40:35.129430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec1caa-9510-4ffb-ad45-50e054023978 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:35.225882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec1caa-9510-4ffb-ad45-50e054023978 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:35.257903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4c6b1-e282-499e-88a9-004b88dc75a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:35.258677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4c6b1-e282-499e-88a9-004b88dc75a3 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:35.259004 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4c6b1-e282-499e-88a9-004b88dc75a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:35.259281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4c6b1-e282-499e-88a9-004b88dc75a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:35.269339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4c6b1-e282-499e-88a9-004b88dc75a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-dde4c6b1-e282-499e-88a9-004b88dc75a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:40:35.276824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:40:35.277914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:40:35.278894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:40:35.278894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:40:35.278894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:40:35.278894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:40:35.278894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:40:35.278894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:40:35.278894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:40:35.278894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:40:35.278894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:40:35.278894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:40:35.278894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:40:35.278894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:40:35.278894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 593/1195] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:40:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:40:35.283894 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a4096-8459-47e8-b434-908f1cb71b57 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:35.375930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a4096-8459-47e8-b434-908f1cb71b57 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:35.407262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bddc21-38f3-4387-95e7-bb7938620af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:35.407912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bddc21-38f3-4387-95e7-bb7938620af2 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:35.408192 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bddc21-38f3-4387-95e7-bb7938620af2 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:35.408466 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bddc21-38f3-4387-95e7-bb7938620af2 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:35.409194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bddc21-38f3-4387-95e7-bb7938620af2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:40:35.409497 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bddc21-38f3-4387-95e7-bb7938620af2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:35.410921 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bddc21-38f3-4387-95e7-bb7938620af2 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:35.419055 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 603/1196] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:40:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:35.424554 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4c6b1-e282-499e-88a9-004b88dc75a3 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:35.518775 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4c6b1-e282-499e-88a9-004b88dc75a3 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:35.551525 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a380b-4f76-4dac-a254-7ec4e527790a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:35.552319 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a380b-4f76-4dac-a254-7ec4e527790a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:35.552615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a380b-4f76-4dac-a254-7ec4e527790a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:35.552880 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a380b-4f76-4dac-a254-7ec4e527790a None admin] PATH_INFO: `/v3/users/0d311b95f53e47afae209a2c8f8d4070` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:35.554077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a380b-4f76-4dac-a254-7ec4e527790a None admin] RBAC: Authorizing `identity:get_user(user_id=0d311b95f53e47afae209a2c8f8d4070)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:35.555546 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a380b-4f76-4dac-a254-7ec4e527790a None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:35.557501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 594/1197] 10.222.0.35 () {56 vars in 1177 bytes} [Fri May 13 21:40:35 2022] GET /identity/v3/users/0d311b95f53e47afae209a2c8f8d4070 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:35.562984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bddc21-38f3-4387-95e7-bb7938620af2 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:35.656201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bddc21-38f3-4387-95e7-bb7938620af2 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:35.687274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22f69f-2667-4409-b31b-b12ad0e3fd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:35.687903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22f69f-2667-4409-b31b-b12ad0e3fd0c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:35.688184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22f69f-2667-4409-b31b-b12ad0e3fd0c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:35.688456 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22f69f-2667-4409-b31b-b12ad0e3fd0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:35.750184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22f69f-2667-4409-b31b-b12ad0e3fd0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:35.751902 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22f69f-2667-4409-b31b-b12ad0e3fd0c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:35.842835 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 604/1198] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:40:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:40:35.848469 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a380b-4f76-4dac-a254-7ec4e527790a None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:35.943536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a380b-4f76-4dac-a254-7ec4e527790a None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:35.975139 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39280b7-99c5-42d3-9891-b0c2cce1c66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:35.975978 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39280b7-99c5-42d3-9891-b0c2cce1c66b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:35.976261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39280b7-99c5-42d3-9891-b0c2cce1c66b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:35.976535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39280b7-99c5-42d3-9891-b0c2cce1c66b None admin] PATH_INFO: `/v3/projects/d5e41cb798df4b8699eaee70b82b7176` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:35.977703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39280b7-99c5-42d3-9891-b0c2cce1c66b None admin] RBAC: Authorizing `identity:get_project(project_id=d5e41cb798df4b8699eaee70b82b7176)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:35.979672 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39280b7-99c5-42d3-9891-b0c2cce1c66b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:35.981367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 595/1199] 10.222.0.35 () {56 vars in 1186 bytes} [Fri May 13 21:40:35 2022] GET /identity/v3/projects/d5e41cb798df4b8699eaee70b82b7176 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:40:35.987490 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22f69f-2667-4409-b31b-b12ad0e3fd0c None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:40:36.084114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22f69f-2667-4409-b31b-b12ad0e3fd0c None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:40:36.115276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f167029-92ee-4dce-b076-8f5b2d573c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:40:36.115848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f167029-92ee-4dce-b076-8f5b2d573c1c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:40:36.116128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f167029-92ee-4dce-b076-8f5b2d573c1c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:40:36.116402 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f167029-92ee-4dce-b076-8f5b2d573c1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:40:36.117212 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f167029-92ee-4dce-b076-8f5b2d573c1c None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=d5e41cb798df4b8699eaee70b82b7176, user.id=0d311b95f53e47afae209a2c8f8d4070) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:40:36.117520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f167029-92ee-4dce-b076-8f5b2d573c1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:40:36.118960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f167029-92ee-4dce-b076-8f5b2d573c1c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:40:36.124984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 605/1200] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:40:35 2022] GET /identity/v3/role_assignments?user.id=0d311b95f53e47afae209a2c8f8d4070&scope.project.id=d5e41cb798df4b8699eaee70b82b7176&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:36.294007 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfd5a9-8ff8-4274-a6b0-d12e03c9742f None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:36.294670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfd5a9-8ff8-4274-a6b0-d12e03c9742f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:36.294670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfd5a9-8ff8-4274-a6b0-d12e03c9742f None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:36.296066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 596/1201] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:41:36.301929 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c9309-7910-48e8-b99f-c7276e075bb6 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:36.302402 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c9309-7910-48e8-b99f-c7276e075bb6 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:36.302504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c9309-7910-48e8-b99f-c7276e075bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:36.322888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-c07c9309-7910-48e8-b99f-c7276e075bb6 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:41:36.328413 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c07c9309-7910-48e8-b99f-c7276e075bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:41:36.424729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 606/1202] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:41:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:41:36.451892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfd5a9-8ff8-4274-a6b0-d12e03c9742f None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:36.584076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfd5a9-8ff8-4274-a6b0-d12e03c9742f None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:36.633640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095949f-e99a-472c-80a0-431563e74eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:36.634185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095949f-e99a-472c-80a0-431563e74eb1 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:36.634491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095949f-e99a-472c-80a0-431563e74eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:36.634792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095949f-e99a-472c-80a0-431563e74eb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:36.647206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095949f-e99a-472c-80a0-431563e74eb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:36.649691 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095949f-e99a-472c-80a0-431563e74eb1 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-2095949f-e99a-472c-80a0-431563e74eb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:41:36.673638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:36.675075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:36.681019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:41:36.681019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:41:36.681019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:36.681019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:41:36.681019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 597/1203] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:36.681350 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c9309-7910-48e8-b99f-c7276e075bb6 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:36.809656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c9309-7910-48e8-b99f-c7276e075bb6 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:36.848102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572fb68-5913-4329-b912-9ec159f095c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:36.848910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572fb68-5913-4329-b912-9ec159f095c8 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:36.849263 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572fb68-5913-4329-b912-9ec159f095c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:36.849629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572fb68-5913-4329-b912-9ec159f095c8 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:36.850514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572fb68-5913-4329-b912-9ec159f095c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:36.850857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572fb68-5913-4329-b912-9ec159f095c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:36.853509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572fb68-5913-4329-b912-9ec159f095c8 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:36.866093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 607/1204] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:41:36.872556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095949f-e99a-472c-80a0-431563e74eb1 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:37.008346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095949f-e99a-472c-80a0-431563e74eb1 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:37.047376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c94bf-e722-4061-8099-1241589aadef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:37.048423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c94bf-e722-4061-8099-1241589aadef None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:37.048936 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c94bf-e722-4061-8099-1241589aadef None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:37.049496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c94bf-e722-4061-8099-1241589aadef None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:37.050430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c94bf-e722-4061-8099-1241589aadef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:37.052179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c94bf-e722-4061-8099-1241589aadef None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:37.078288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 598/1205] 10.222.0.35 () {60 vars in 1132 bytes} [Fri May 13 21:41:36 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:41:39.511894 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a07485-8ff3-4469-9cf1-dec2df5934e3 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:39.512498 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a07485-8ff3-4469-9cf1-dec2df5934e3 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:39.512498 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a07485-8ff3-4469-9cf1-dec2df5934e3 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:39.513838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 608/1206] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:41:39.519237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce094748-8128-46df-a5d1-d5f48331d116 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:39.519750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce094748-8128-46df-a5d1-d5f48331d116 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:39.519852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce094748-8128-46df-a5d1-d5f48331d116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:39.538093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ce094748-8128-46df-a5d1-d5f48331d116 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:41:39.539362 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ce094748-8128-46df-a5d1-d5f48331d116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:41:39.612664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 599/1207] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:41:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:41:39.646881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a07485-8ff3-4469-9cf1-dec2df5934e3 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:39.758102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a07485-8ff3-4469-9cf1-dec2df5934e3 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:39.793272 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa8872-31d4-437b-ad05-acafd1e8f83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:39.794097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa8872-31d4-437b-ad05-acafd1e8f83c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:39.794380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa8872-31d4-437b-ad05-acafd1e8f83c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:39.794626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa8872-31d4-437b-ad05-acafd1e8f83c None admin] PATH_INFO: `/v3/roles/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:39.806991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa8872-31d4-437b-ad05-acafd1e8f83c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-6cfa8872-31d4-437b-ad05-acafd1e8f83c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:41:39.816747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:39.817981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:39.819079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:39.819079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:39.819079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:39.819079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:39.819079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:41:39.819079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:41:39.819079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:41:39.819079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:41:39.819079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:41:39.819079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:41:39.819079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:41:39.819079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:41:39.826341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 609/1208] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:41:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:39.829930 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce094748-8128-46df-a5d1-d5f48331d116 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:39.945653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce094748-8128-46df-a5d1-d5f48331d116 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:39.983200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00b09-cccd-40b6-a03b-7d7435f65af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:39.984372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00b09-cccd-40b6-a03b-7d7435f65af2 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:39.984743 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00b09-cccd-40b6-a03b-7d7435f65af2 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:39.985030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00b09-cccd-40b6-a03b-7d7435f65af2 None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:39.985818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a00b09-cccd-40b6-a03b-7d7435f65af2 None admin] RBAC: Adding query filter params (name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:39.986180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a00b09-cccd-40b6-a03b-7d7435f65af2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:39.987872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a00b09-cccd-40b6-a03b-7d7435f65af2 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:40.002524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 600/1209] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:41:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:40.004012 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa8872-31d4-437b-ad05-acafd1e8f83c None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:40.118870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa8872-31d4-437b-ad05-acafd1e8f83c None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:40.157731 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74892ebd-9e11-4fdf-a9de-a4c0b8e44580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:40.158530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74892ebd-9e11-4fdf-a9de-a4c0b8e44580 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:40.158845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74892ebd-9e11-4fdf-a9de-a4c0b8e44580 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:40.159128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74892ebd-9e11-4fdf-a9de-a4c0b8e44580 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:40.170207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74892ebd-9e11-4fdf-a9de-a4c0b8e44580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:40.171784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74892ebd-9e11-4fdf-a9de-a4c0b8e44580 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-74892ebd-9e11-4fdf-a9de-a4c0b8e44580 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:41:40.193457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:40.194830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:40.196688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:41:40.196688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:41:40.196688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:40.196688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:41:40.200633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 610/1210] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:41:40.206606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00b09-cccd-40b6-a03b-7d7435f65af2 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:40.323094 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00b09-cccd-40b6-a03b-7d7435f65af2 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:40.363724 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb482b-e4fe-4a8f-8182-c37d549f86c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:40.364404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb482b-e4fe-4a8f-8182-c37d549f86c2 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:40.364689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb482b-e4fe-4a8f-8182-c37d549f86c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:40.364961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb482b-e4fe-4a8f-8182-c37d549f86c2 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:40.365695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb482b-e4fe-4a8f-8182-c37d549f86c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:40.366070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb482b-e4fe-4a8f-8182-c37d549f86c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:40.369405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb482b-e4fe-4a8f-8182-c37d549f86c2 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:40.381354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 601/1211] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:40.387457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74892ebd-9e11-4fdf-a9de-a4c0b8e44580 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:40.494571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74892ebd-9e11-4fdf-a9de-a4c0b8e44580 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:40.533326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55d19b-8efe-44c2-8a37-9fe0b12d85d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:40.534441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55d19b-8efe-44c2-8a37-9fe0b12d85d8 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:40.534441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55d19b-8efe-44c2-8a37-9fe0b12d85d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:40.534693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55d19b-8efe-44c2-8a37-9fe0b12d85d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:40.543642 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55d19b-8efe-44c2-8a37-9fe0b12d85d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:40.545346 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55d19b-8efe-44c2-8a37-9fe0b12d85d8 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-cb55d19b-8efe-44c2-8a37-9fe0b12d85d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:41:40.564227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:40.565591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:40.566555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:40.566555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:40.566555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:41:40.566555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:41:40.566555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:41:40.566555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:41:40.566555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:41:40.566555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:41:40.566555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 21:41:40.566555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:41:40.566555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 611/1212] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:41:40.571780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb482b-e4fe-4a8f-8182-c37d549f86c2 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:40.703042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb482b-e4fe-4a8f-8182-c37d549f86c2 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:40.742968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d3d19-423c-4c9a-a073-2349529b65c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:40.743710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d3d19-423c-4c9a-a073-2349529b65c4 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:40.743962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d3d19-423c-4c9a-a073-2349529b65c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:40.744246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d3d19-423c-4c9a-a073-2349529b65c4 None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:40.745287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d3d19-423c-4c9a-a073-2349529b65c4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:40.745603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d3d19-423c-4c9a-a073-2349529b65c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:40.747153 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d3d19-423c-4c9a-a073-2349529b65c4 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:40.770195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 602/1213] 10.222.0.35 () {56 vars in 1143 bytes} [Fri May 13 21:41:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:40.771704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55d19b-8efe-44c2-8a37-9fe0b12d85d8 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:40.874769 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55d19b-8efe-44c2-8a37-9fe0b12d85d8 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:40.909949 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4884092-f962-456d-a550-2290aa034201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:40.910632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4884092-f962-456d-a550-2290aa034201 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:40.910935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4884092-f962-456d-a550-2290aa034201 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:40.911237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4884092-f962-456d-a550-2290aa034201 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:40.980148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4884092-f962-456d-a550-2290aa034201 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:40.981521 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4884092-f962-456d-a550-2290aa034201 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:41.098312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 612/1214] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:41:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:41:41.099712 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d3d19-423c-4c9a-a073-2349529b65c4 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:41.207451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d3d19-423c-4c9a-a073-2349529b65c4 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:41.244020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d22bbf-6eda-42bc-b9a8-7070359e2862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:41.244713 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d22bbf-6eda-42bc-b9a8-7070359e2862 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:41.244991 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d22bbf-6eda-42bc-b9a8-7070359e2862 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:41.245258 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d22bbf-6eda-42bc-b9a8-7070359e2862 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:41.255446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d22bbf-6eda-42bc-b9a8-7070359e2862 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:41.256947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d22bbf-6eda-42bc-b9a8-7070359e2862 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-f8d22bbf-6eda-42bc-b9a8-7070359e2862 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:41:41.276902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:41.279799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:41.280694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:41:41.280694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:41:41.280694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:41.280694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:41:41.280694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 603/1215] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:41.283824 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4884092-f962-456d-a550-2290aa034201 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:41.394093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4884092-f962-456d-a550-2290aa034201 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:41.430880 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf35eb-607d-4b3b-89bb-bcb7c79afc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:41.431545 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf35eb-607d-4b3b-89bb-bcb7c79afc74 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:41.431831 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf35eb-607d-4b3b-89bb-bcb7c79afc74 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:41.432107 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf35eb-607d-4b3b-89bb-bcb7c79afc74 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:41.432967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf35eb-607d-4b3b-89bb-bcb7c79afc74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:41.433283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf35eb-607d-4b3b-89bb-bcb7c79afc74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:41.435177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf35eb-607d-4b3b-89bb-bcb7c79afc74 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:41.449681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 613/1216] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:41.455443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d22bbf-6eda-42bc-b9a8-7070359e2862 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:41.565404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d22bbf-6eda-42bc-b9a8-7070359e2862 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:41.606318 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1a983-44ea-40a8-95e1-feb540d33f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:41.607109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1a983-44ea-40a8-95e1-feb540d33f46 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:41.607392 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1a983-44ea-40a8-95e1-feb540d33f46 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:41.607668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1a983-44ea-40a8-95e1-feb540d33f46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:41.615392 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1a983-44ea-40a8-95e1-feb540d33f46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:41.617412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1a983-44ea-40a8-95e1-feb540d33f46 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-e7a1a983-44ea-40a8-95e1-feb540d33f46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:41:41.632076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:41.633935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:41.635202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:41:41.635202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:41:41.635202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:41:41.635202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:41:41.635202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:41:41.635202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:41:41.635202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:41:41.635202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:41:41.638738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 604/1217] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:41:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:41.639781 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf35eb-607d-4b3b-89bb-bcb7c79afc74 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:41.759829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf35eb-607d-4b3b-89bb-bcb7c79afc74 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:41.798931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7f9b4-8e63-4db6-a888-487d260a94c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:41.799763 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7f9b4-8e63-4db6-a888-487d260a94c6 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:41.800065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7f9b4-8e63-4db6-a888-487d260a94c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:41.800475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7f9b4-8e63-4db6-a888-487d260a94c6 None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:41.801491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7f9b4-8e63-4db6-a888-487d260a94c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:41.801912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7f9b4-8e63-4db6-a888-487d260a94c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:41.803823 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7f9b4-8e63-4db6-a888-487d260a94c6 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:41.816911 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 614/1218] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:41:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:41.822584 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1a983-44ea-40a8-95e1-feb540d33f46 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:41.931209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1a983-44ea-40a8-95e1-feb540d33f46 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:41.968046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efa50b-953e-4370-a3fb-efcb9ac2d53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:41.968746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efa50b-953e-4370-a3fb-efcb9ac2d53c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:41.969000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efa50b-953e-4370-a3fb-efcb9ac2d53c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:41.969314 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efa50b-953e-4370-a3fb-efcb9ac2d53c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:41.970261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7efa50b-953e-4370-a3fb-efcb9ac2d53c None admin] RBAC: Adding query filter params (role.id=deb38f497be14e78ae21e8dceca3772f, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=8124ab8abcc640c5a48b3e0216afa48d) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:41.970571 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7efa50b-953e-4370-a3fb-efcb9ac2d53c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:41.972070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7efa50b-953e-4370-a3fb-efcb9ac2d53c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:41.978920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 605/1219] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:41:41 2022] GET /identity/v3/role_assignments?user.id=8124ab8abcc640c5a48b3e0216afa48d&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=deb38f497be14e78ae21e8dceca3772f => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:44.381203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acfe8f-e923-4915-a19f-5e860339e31e None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:44.381789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acfe8f-e923-4915-a19f-5e860339e31e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:44.381789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acfe8f-e923-4915-a19f-5e860339e31e None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:44.386754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 615/1220] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:41:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:41:44.390025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf287-08b9-40bf-a4f1-6936d642e4ac None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:44.390570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf287-08b9-40bf-a4f1-6936d642e4ac None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:44.390802 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf287-08b9-40bf-a4f1-6936d642e4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:44.409868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-492bf287-08b9-40bf-a4f1-6936d642e4ac None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:41:44.411526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-492bf287-08b9-40bf-a4f1-6936d642e4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:41:44.484560 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 606/1221] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:41:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:41:44.508640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acfe8f-e923-4915-a19f-5e860339e31e None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:44.613235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acfe8f-e923-4915-a19f-5e860339e31e None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:44.650176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2c83a-e04a-43be-8536-56a7c44f7a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:44.650995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2c83a-e04a-43be-8536-56a7c44f7a95 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:44.651275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2c83a-e04a-43be-8536-56a7c44f7a95 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:44.651546 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2c83a-e04a-43be-8536-56a7c44f7a95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:44.661517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2c83a-e04a-43be-8536-56a7c44f7a95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-c6f2c83a-e04a-43be-8536-56a7c44f7a95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:41:44.669078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:44.670052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:44.670957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:44.670957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:44.670957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:44.670957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:44.670957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:41:44.670957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:41:44.670957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:41:44.670957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:41:44.670957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:41:44.670957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:41:44.670957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:41:44.670957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:41:44.670957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 616/1222] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:41:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:41:44.676774 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf287-08b9-40bf-a4f1-6936d642e4ac None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:44.786868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf287-08b9-40bf-a4f1-6936d642e4ac None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:44.823427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffeb35-34c3-474e-aeeb-7ae4f1d8cabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:44.824322 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffeb35-34c3-474e-aeeb-7ae4f1d8cabc None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:44.824603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffeb35-34c3-474e-aeeb-7ae4f1d8cabc None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:44.824871 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffeb35-34c3-474e-aeeb-7ae4f1d8cabc None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:44.825604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffeb35-34c3-474e-aeeb-7ae4f1d8cabc None admin] RBAC: Adding query filter params (name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:44.825900 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffeb35-34c3-474e-aeeb-7ae4f1d8cabc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:44.827292 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffeb35-34c3-474e-aeeb-7ae4f1d8cabc None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:44.835003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 607/1223] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:41:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:44.841196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2c83a-e04a-43be-8536-56a7c44f7a95 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:44.946387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2c83a-e04a-43be-8536-56a7c44f7a95 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:44.979733 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c10415-b943-44d8-b288-de2c6e440442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:44.980515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c10415-b943-44d8-b288-de2c6e440442 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:44.980819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c10415-b943-44d8-b288-de2c6e440442 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:44.981106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c10415-b943-44d8-b288-de2c6e440442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:44.991546 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c10415-b943-44d8-b288-de2c6e440442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:44.993559 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c10415-b943-44d8-b288-de2c6e440442 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-02c10415-b943-44d8-b288-de2c6e440442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:41:45.010811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:45.011788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:45.013124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:41:45.013124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:41:45.013124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:45.013124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:41:45.013124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 617/1224] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:41:45.017826 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffeb35-34c3-474e-aeeb-7ae4f1d8cabc None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:45.119977 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffeb35-34c3-474e-aeeb-7ae4f1d8cabc None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:45.156911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075898c8-800c-4fa8-b6f4-774cbbcadf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:45.157691 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075898c8-800c-4fa8-b6f4-774cbbcadf65 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:45.158011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075898c8-800c-4fa8-b6f4-774cbbcadf65 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:45.158319 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075898c8-800c-4fa8-b6f4-774cbbcadf65 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:45.159164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075898c8-800c-4fa8-b6f4-774cbbcadf65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:45.159531 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075898c8-800c-4fa8-b6f4-774cbbcadf65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:45.161097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075898c8-800c-4fa8-b6f4-774cbbcadf65 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:45.174471 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 608/1225] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:45.180172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c10415-b943-44d8-b288-de2c6e440442 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:45.292477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c10415-b943-44d8-b288-de2c6e440442 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:45.338616 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f79d6-9b49-47a8-9cf9-e7b0382d4c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:45.339445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f79d6-9b49-47a8-9cf9-e7b0382d4c82 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:45.339727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f79d6-9b49-47a8-9cf9-e7b0382d4c82 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:45.340002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f79d6-9b49-47a8-9cf9-e7b0382d4c82 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:45.347874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f79d6-9b49-47a8-9cf9-e7b0382d4c82 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:45.349341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f79d6-9b49-47a8-9cf9-e7b0382d4c82 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-880f79d6-9b49-47a8-9cf9-e7b0382d4c82 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:41:45.364725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:45.365873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:45.366867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:45.366867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:45.366867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:41:45.366867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:41:45.366867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:41:45.366867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:41:45.366867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:41:45.366867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:41:45.366867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 21:41:45.366867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:41:45.366867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 618/1226] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:45.371935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075898c8-800c-4fa8-b6f4-774cbbcadf65 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:45.476125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075898c8-800c-4fa8-b6f4-774cbbcadf65 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:45.509363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b895579-91d9-47ae-baaf-e1a0c4dac3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:45.510026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b895579-91d9-47ae-baaf-e1a0c4dac3ae None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:45.510305 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b895579-91d9-47ae-baaf-e1a0c4dac3ae None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:45.510572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b895579-91d9-47ae-baaf-e1a0c4dac3ae None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:45.511431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b895579-91d9-47ae-baaf-e1a0c4dac3ae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:45.511728 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b895579-91d9-47ae-baaf-e1a0c4dac3ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:45.513187 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b895579-91d9-47ae-baaf-e1a0c4dac3ae None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:45.530474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 609/1227] 10.222.0.35 () {56 vars in 1143 bytes} [Fri May 13 21:41:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:45.536158 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f79d6-9b49-47a8-9cf9-e7b0382d4c82 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:45.650590 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f79d6-9b49-47a8-9cf9-e7b0382d4c82 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:45.683931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bc4cd-831c-407e-8485-027fbfdd2a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:45.684802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bc4cd-831c-407e-8485-027fbfdd2a68 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:45.685091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bc4cd-831c-407e-8485-027fbfdd2a68 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:45.685360 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bc4cd-831c-407e-8485-027fbfdd2a68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:45.697429 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bc4cd-831c-407e-8485-027fbfdd2a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:45.699130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bc4cd-831c-407e-8485-027fbfdd2a68 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-3e7bc4cd-831c-407e-8485-027fbfdd2a68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:41:45.717810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:45.718971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:45.724844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:41:45.724844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:41:45.724844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:45.724844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:41:45.724844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 619/1228] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:41:45.726994 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b895579-91d9-47ae-baaf-e1a0c4dac3ae None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:45.836055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b895579-91d9-47ae-baaf-e1a0c4dac3ae None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:45.869869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d729dcd-e24a-484b-880f-f254d0dc26cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:45.870539 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d729dcd-e24a-484b-880f-f254d0dc26cd None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:45.870867 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d729dcd-e24a-484b-880f-f254d0dc26cd None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:45.871148 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d729dcd-e24a-484b-880f-f254d0dc26cd None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:45.871864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d729dcd-e24a-484b-880f-f254d0dc26cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:45.872169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d729dcd-e24a-484b-880f-f254d0dc26cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:45.873546 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d729dcd-e24a-484b-880f-f254d0dc26cd None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:45.886184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 610/1229] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:45.891930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bc4cd-831c-407e-8485-027fbfdd2a68 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:46.005637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bc4cd-831c-407e-8485-027fbfdd2a68 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:46.046034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf676a5-9dc7-453c-82fc-12233dc44164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:46.046988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf676a5-9dc7-453c-82fc-12233dc44164 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:46.047393 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf676a5-9dc7-453c-82fc-12233dc44164 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:46.047796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf676a5-9dc7-453c-82fc-12233dc44164 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:46.056070 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf676a5-9dc7-453c-82fc-12233dc44164 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:46.057517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf676a5-9dc7-453c-82fc-12233dc44164 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-1cf676a5-9dc7-453c-82fc-12233dc44164 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:41:46.073619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:46.074802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:46.075734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:41:46.075734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:41:46.075734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:41:46.075734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:41:46.075734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:41:46.075734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:41:46.075734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:41:46.075734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:41:46.075734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 620/1230] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:41:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:46.080593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d729dcd-e24a-484b-880f-f254d0dc26cd None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:46.182123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d729dcd-e24a-484b-880f-f254d0dc26cd None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:46.219285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f3046-b095-4e7a-9f59-c812b2b81596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:46.219944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f3046-b095-4e7a-9f59-c812b2b81596 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:46.220225 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f3046-b095-4e7a-9f59-c812b2b81596 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:46.220500 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f3046-b095-4e7a-9f59-c812b2b81596 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:46.221295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f3046-b095-4e7a-9f59-c812b2b81596 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:46.221597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f3046-b095-4e7a-9f59-c812b2b81596 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:46.223071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f3046-b095-4e7a-9f59-c812b2b81596 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:46.235794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 611/1231] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:41:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:46.242314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf676a5-9dc7-453c-82fc-12233dc44164 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:46.352033 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf676a5-9dc7-453c-82fc-12233dc44164 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:46.388290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65eb1d1-6b04-496a-9f26-ee999dfb3f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:46.389244 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eb1d1-6b04-496a-9f26-ee999dfb3f7b None admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:46.389570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eb1d1-6b04-496a-9f26-ee999dfb3f7b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:46.389857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eb1d1-6b04-496a-9f26-ee999dfb3f7b None admin] PATH_INFO: `/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/8124ab8abcc640c5a48b3e0216afa48d/roles/deb38f497be14e78ae21e8dceca3772f` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:46.405146 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65eb1d1-6b04-496a-9f26-ee999dfb3f7b None admin] RBAC: Authorizing `identity:create_grant(project_id=ac834e84b2c84261ade311a6dfbbeb20, user_id=8124ab8abcc640c5a48b3e0216afa48d, role_id=deb38f497be14e78ae21e8dceca3772f)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:46.406784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65eb1d1-6b04-496a-9f26-ee999dfb3f7b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:46.419060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 621/1232] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:41:46 2022] PUT /identity/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/8124ab8abcc640c5a48b3e0216afa48d/roles/deb38f497be14e78ae21e8dceca3772f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:41:48.741967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d756a8e-334b-44f2-855e-fcb40e76849b None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:48.742548 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d756a8e-334b-44f2-855e-fcb40e76849b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:48.742637 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d756a8e-334b-44f2-855e-fcb40e76849b None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:48.744493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 612/1233] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:41:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:41:48.750417 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175044a5-3374-469a-80f9-69f7789910ba None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:48.750878 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175044a5-3374-469a-80f9-69f7789910ba None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:48.751011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175044a5-3374-469a-80f9-69f7789910ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:48.768302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-175044a5-3374-469a-80f9-69f7789910ba None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:41:48.769686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-175044a5-3374-469a-80f9-69f7789910ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:41:48.843195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 622/1234] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:41:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:41:48.867163 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d756a8e-334b-44f2-855e-fcb40e76849b None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:48.978525 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d756a8e-334b-44f2-855e-fcb40e76849b None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:49.013116 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cc851-d6b8-4fad-9102-605d733e998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:49.013835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cc851-d6b8-4fad-9102-605d733e998e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:49.014112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cc851-d6b8-4fad-9102-605d733e998e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:49.014390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cc851-d6b8-4fad-9102-605d733e998e None admin] PATH_INFO: `/v3/roles/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:49.025857 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6cc851-d6b8-4fad-9102-605d733e998e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-1e6cc851-d6b8-4fad-9102-605d733e998e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:41:49.033925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:49.035143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:49.041530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:49.041530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:49.041530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:49.041530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:49.041530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:41:49.041530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:41:49.041530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:41:49.041530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:41:49.041530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:41:49.041530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:41:49.041530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 21:41:49.041530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:41:49.041530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 613/1235] 10.222.0.35 () {56 vars in 1102 bytes} [Fri May 13 21:41:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:49.042382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175044a5-3374-469a-80f9-69f7789910ba None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:49.144663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175044a5-3374-469a-80f9-69f7789910ba None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:49.182280 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f842619-1019-462e-966a-33975a6f4713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:49.182976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f842619-1019-462e-966a-33975a6f4713 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:49.183258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f842619-1019-462e-966a-33975a6f4713 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:49.183536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f842619-1019-462e-966a-33975a6f4713 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:49.184268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f842619-1019-462e-966a-33975a6f4713 None admin] RBAC: Adding query filter params (name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:49.184570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f842619-1019-462e-966a-33975a6f4713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:49.185936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f842619-1019-462e-966a-33975a6f4713 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:49.195918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 623/1236] 10.222.0.35 () {56 vars in 1103 bytes} [Fri May 13 21:41:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:41:49.201500 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cc851-d6b8-4fad-9102-605d733e998e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:49.310183 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cc851-d6b8-4fad-9102-605d733e998e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:49.345533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423bef4-2d7a-410a-b160-64818ae436d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:49.346381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423bef4-2d7a-410a-b160-64818ae436d0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:49.346652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423bef4-2d7a-410a-b160-64818ae436d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:49.346954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423bef4-2d7a-410a-b160-64818ae436d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:49.358444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423bef4-2d7a-410a-b160-64818ae436d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:49.359955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423bef4-2d7a-410a-b160-64818ae436d0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-a423bef4-2d7a-410a-b160-64818ae436d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:41:49.376807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:49.377865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:49.378876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:41:49.378876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:41:49.378876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:49.378876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:41:49.378876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 614/1237] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:49.384235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f842619-1019-462e-966a-33975a6f4713 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:49.488996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f842619-1019-462e-966a-33975a6f4713 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:49.526973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521370e-001b-4eb7-84e1-bef5c6f8c1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:49.527657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521370e-001b-4eb7-84e1-bef5c6f8c1e5 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:49.527935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521370e-001b-4eb7-84e1-bef5c6f8c1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:49.528203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521370e-001b-4eb7-84e1-bef5c6f8c1e5 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:49.528940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521370e-001b-4eb7-84e1-bef5c6f8c1e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:49.529288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521370e-001b-4eb7-84e1-bef5c6f8c1e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:49.530727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521370e-001b-4eb7-84e1-bef5c6f8c1e5 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:49.542855 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 624/1238] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:49.548486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423bef4-2d7a-410a-b160-64818ae436d0 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:49.659428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423bef4-2d7a-410a-b160-64818ae436d0 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:49.695971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b8cda-7457-4b61-a598-a7d05bd21b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:49.696785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b8cda-7457-4b61-a598-a7d05bd21b89 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:49.697065 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b8cda-7457-4b61-a598-a7d05bd21b89 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:49.697337 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b8cda-7457-4b61-a598-a7d05bd21b89 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:49.704957 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4b8cda-7457-4b61-a598-a7d05bd21b89 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:49.706402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4b8cda-7457-4b61-a598-a7d05bd21b89 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-ab4b8cda-7457-4b61-a598-a7d05bd21b89 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:41:49.720676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:49.721692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:49.722847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:49.722847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:49.722847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:41:49.722847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:41:49.722847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:41:49.722847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:41:49.722847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:41:49.722847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:41:49.722847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 21:41:49.722847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:41:49.722847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 615/1239] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:49.727925 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521370e-001b-4eb7-84e1-bef5c6f8c1e5 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:49.838801 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521370e-001b-4eb7-84e1-bef5c6f8c1e5 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:49.873621 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd80cb-b5c9-426c-8e45-922980f2b018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:49.874597 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd80cb-b5c9-426c-8e45-922980f2b018 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:49.874901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd80cb-b5c9-426c-8e45-922980f2b018 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:49.875182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd80cb-b5c9-426c-8e45-922980f2b018 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:49.876182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd80cb-b5c9-426c-8e45-922980f2b018 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:49.876581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd80cb-b5c9-426c-8e45-922980f2b018 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:49.878245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd80cb-b5c9-426c-8e45-922980f2b018 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:49.898995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 625/1240] 10.222.0.35 () {56 vars in 1143 bytes} [Fri May 13 21:41:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:49.904667 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b8cda-7457-4b61-a598-a7d05bd21b89 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:50.014515 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b8cda-7457-4b61-a598-a7d05bd21b89 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:50.048418 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35895095-1923-4b6d-a506-d572c53519c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:50.049064 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35895095-1923-4b6d-a506-d572c53519c1 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:50.049343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35895095-1923-4b6d-a506-d572c53519c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:50.049613 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35895095-1923-4b6d-a506-d572c53519c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:50.117146 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35895095-1923-4b6d-a506-d572c53519c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:50.119470 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35895095-1923-4b6d-a506-d572c53519c1 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:50.224300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 616/1241] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:41:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 21:41:50.231766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd80cb-b5c9-426c-8e45-922980f2b018 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:50.342664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd80cb-b5c9-426c-8e45-922980f2b018 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:50.379600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ac3be-f8db-4ca8-a428-f6cf827b2d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:50.380379 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ac3be-f8db-4ca8-a428-f6cf827b2d3c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:50.380658 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ac3be-f8db-4ca8-a428-f6cf827b2d3c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:50.380937 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ac3be-f8db-4ca8-a428-f6cf827b2d3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:50.391320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ac3be-f8db-4ca8-a428-f6cf827b2d3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:50.392902 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ac3be-f8db-4ca8-a428-f6cf827b2d3c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-a70ac3be-f8db-4ca8-a428-f6cf827b2d3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:41:50.410681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:50.411743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:50.412655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:41:50.412655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:41:50.412655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:50.412655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:41:50.414243 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 626/1242] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:50.419822 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35895095-1923-4b6d-a506-d572c53519c1 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:50.518377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35895095-1923-4b6d-a506-d572c53519c1 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:50.551941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442e358-7231-493d-962d-ea3fc33a939b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:50.552615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442e358-7231-493d-962d-ea3fc33a939b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:50.552862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442e358-7231-493d-962d-ea3fc33a939b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:50.553136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442e358-7231-493d-962d-ea3fc33a939b None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:50.553946 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442e358-7231-493d-962d-ea3fc33a939b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:50.554907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442e358-7231-493d-962d-ea3fc33a939b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:50.556306 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442e358-7231-493d-962d-ea3fc33a939b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:50.571008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 617/1243] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:41:50.576786 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ac3be-f8db-4ca8-a428-f6cf827b2d3c None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:50.686804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ac3be-f8db-4ca8-a428-f6cf827b2d3c None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:50.723260 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9d768-b952-4abc-b5e4-778a43f2e36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:50.724054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9d768-b952-4abc-b5e4-778a43f2e36a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:50.724332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9d768-b952-4abc-b5e4-778a43f2e36a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:50.724602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9d768-b952-4abc-b5e4-778a43f2e36a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:50.732439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9d768-b952-4abc-b5e4-778a43f2e36a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:50.734032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9d768-b952-4abc-b5e4-778a43f2e36a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-baf9d768-b952-4abc-b5e4-778a43f2e36a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:41:50.748353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:50.749861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:50.750816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:41:50.750816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:41:50.750816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:41:50.750816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:41:50.750816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:41:50.750816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:41:50.750816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:41:50.750816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:41:50.750816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 627/1244] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:41:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 21:41:50.755745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442e358-7231-493d-962d-ea3fc33a939b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:50.858025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442e358-7231-493d-962d-ea3fc33a939b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:50.892222 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3525d-3ef0-40dc-b1f9-ba719f57598a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:50.892877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3525d-3ef0-40dc-b1f9-ba719f57598a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:50.893161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3525d-3ef0-40dc-b1f9-ba719f57598a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:50.893436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3525d-3ef0-40dc-b1f9-ba719f57598a None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:50.894214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3525d-3ef0-40dc-b1f9-ba719f57598a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:50.894518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3525d-3ef0-40dc-b1f9-ba719f57598a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:50.896001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3525d-3ef0-40dc-b1f9-ba719f57598a None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:50.908630 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 618/1245] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:41:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:41:50.915538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9d768-b952-4abc-b5e4-778a43f2e36a None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:51.020775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9d768-b952-4abc-b5e4-778a43f2e36a None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:51.055406 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbfc22-51be-40b3-a182-0c028a680b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:51.056091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbfc22-51be-40b3-a182-0c028a680b83 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:51.056367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbfc22-51be-40b3-a182-0c028a680b83 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:51.056676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbfc22-51be-40b3-a182-0c028a680b83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:51.057515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbfc22-51be-40b3-a182-0c028a680b83 None admin] RBAC: Adding query filter params (role.id=deb38f497be14e78ae21e8dceca3772f, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=8124ab8abcc640c5a48b3e0216afa48d) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:51.057832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbfc22-51be-40b3-a182-0c028a680b83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:51.059281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbfc22-51be-40b3-a182-0c028a680b83 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:51.065119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 628/1246] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:41:50 2022] GET /identity/v3/role_assignments?user.id=8124ab8abcc640c5a48b3e0216afa48d&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=deb38f497be14e78ae21e8dceca3772f => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:53.651107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce00c5-cf20-4b9c-9a18-af20e51431c2 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:53.651721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce00c5-cf20-4b9c-9a18-af20e51431c2 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:53.651721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce00c5-cf20-4b9c-9a18-af20e51431c2 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:53.653014 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 619/1247] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:41:53.659124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20677c89-ed35-43a3-aa71-5b7a848cb9f4 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:53.659702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20677c89-ed35-43a3-aa71-5b7a848cb9f4 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:53.659702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20677c89-ed35-43a3-aa71-5b7a848cb9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:53.678166 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-20677c89-ed35-43a3-aa71-5b7a848cb9f4 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:41:53.679659 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-20677c89-ed35-43a3-aa71-5b7a848cb9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:41:53.754104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 629/1248] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:41:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:41:53.780578 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce00c5-cf20-4b9c-9a18-af20e51431c2 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:53.892853 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce00c5-cf20-4b9c-9a18-af20e51431c2 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:53.929239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5351b0-8ca7-4af5-8a32-cd101365602b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:53.930061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5351b0-8ca7-4af5-8a32-cd101365602b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:53.930393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5351b0-8ca7-4af5-8a32-cd101365602b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:53.930735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5351b0-8ca7-4af5-8a32-cd101365602b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:53.940791 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5351b0-8ca7-4af5-8a32-cd101365602b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-4e5351b0-8ca7-4af5-8a32-cd101365602b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:41:53.948923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:53.949940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:53.950923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:53.950923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:53.950923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:53.950923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:53.950923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:41:53.950923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:41:53.950923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:41:53.950923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:41:53.950923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:41:53.950923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:41:53.950923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:41:53.950923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:41:53.950923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 620/1249] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:41:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:53.956008 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20677c89-ed35-43a3-aa71-5b7a848cb9f4 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:54.058679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20677c89-ed35-43a3-aa71-5b7a848cb9f4 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:54.097059 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce636d3-66f8-40a8-b55c-04cbc4bd39d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:54.097762 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce636d3-66f8-40a8-b55c-04cbc4bd39d6 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:54.098071 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce636d3-66f8-40a8-b55c-04cbc4bd39d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:54.098379 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce636d3-66f8-40a8-b55c-04cbc4bd39d6 None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:54.099260 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce636d3-66f8-40a8-b55c-04cbc4bd39d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:54.099621 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce636d3-66f8-40a8-b55c-04cbc4bd39d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:54.101755 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce636d3-66f8-40a8-b55c-04cbc4bd39d6 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:54.110317 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 630/1250] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:41:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:54.116368 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5351b0-8ca7-4af5-8a32-cd101365602b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:54.220621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5351b0-8ca7-4af5-8a32-cd101365602b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:54.257039 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b75115-40d9-4e3c-8994-82fbcfbe1448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:54.258442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b75115-40d9-4e3c-8994-82fbcfbe1448 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:54.258939 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b75115-40d9-4e3c-8994-82fbcfbe1448 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:54.260558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b75115-40d9-4e3c-8994-82fbcfbe1448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:54.271607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b75115-40d9-4e3c-8994-82fbcfbe1448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:54.273191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b75115-40d9-4e3c-8994-82fbcfbe1448 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-e0b75115-40d9-4e3c-8994-82fbcfbe1448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:41:54.293649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:54.294882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:54.295809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:41:54.295809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:41:54.295809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:54.295809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:41:54.295809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 621/1251] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:54.300738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce636d3-66f8-40a8-b55c-04cbc4bd39d6 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:54.402895 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce636d3-66f8-40a8-b55c-04cbc4bd39d6 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:54.436306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f21c8-2e7e-41a1-97a6-084967859f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:54.436962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f21c8-2e7e-41a1-97a6-084967859f67 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:54.437245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f21c8-2e7e-41a1-97a6-084967859f67 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:54.437513 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f21c8-2e7e-41a1-97a6-084967859f67 None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:54.438224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f21c8-2e7e-41a1-97a6-084967859f67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:54.438531 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f21c8-2e7e-41a1-97a6-084967859f67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:54.439948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f21c8-2e7e-41a1-97a6-084967859f67 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:54.451968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 631/1252] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:54.457599 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b75115-40d9-4e3c-8994-82fbcfbe1448 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:54.560543 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b75115-40d9-4e3c-8994-82fbcfbe1448 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:54.598055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2401a4-45ff-4e99-a249-e2606741250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:54.598901 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2401a4-45ff-4e99-a249-e2606741250c None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:54.599198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2401a4-45ff-4e99-a249-e2606741250c None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:54.599428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2401a4-45ff-4e99-a249-e2606741250c None admin] PATH_INFO: `/v3/users/placement` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:54.608196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2401a4-45ff-4e99-a249-e2606741250c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:54.609645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2401a4-45ff-4e99-a249-e2606741250c None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-dd2401a4-45ff-4e99-a249-e2606741250c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:41:54.625120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:54.626232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:54.627334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:54.627334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:54.627334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:41:54.627334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:41:54.627334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:41:54.627334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:41:54.627334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:41:54.627334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:41:54.627334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 21:41:54.627334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:41:54.627334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 622/1253] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:54.632478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f21c8-2e7e-41a1-97a6-084967859f67 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:54.742012 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f21c8-2e7e-41a1-97a6-084967859f67 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:54.777043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394912-42a0-4e82-8f8f-920c3387f656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:54.777720 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394912-42a0-4e82-8f8f-920c3387f656 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:54.778016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394912-42a0-4e82-8f8f-920c3387f656 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:54.778294 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394912-42a0-4e82-8f8f-920c3387f656 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:54.779133 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58394912-42a0-4e82-8f8f-920c3387f656 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:54.779445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58394912-42a0-4e82-8f8f-920c3387f656 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:54.780901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58394912-42a0-4e82-8f8f-920c3387f656 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:54.798671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 632/1254] 10.222.0.35 () {56 vars in 1143 bytes} [Fri May 13 21:41:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:54.804939 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2401a4-45ff-4e99-a249-e2606741250c None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:54.907848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2401a4-45ff-4e99-a249-e2606741250c None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:54.943447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719bf0f-f8b1-4918-b238-950b01f6cca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:54.944105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719bf0f-f8b1-4918-b238-950b01f6cca2 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:54.944392 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719bf0f-f8b1-4918-b238-950b01f6cca2 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:54.944695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719bf0f-f8b1-4918-b238-950b01f6cca2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:55.015527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719bf0f-f8b1-4918-b238-950b01f6cca2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:55.017037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719bf0f-f8b1-4918-b238-950b01f6cca2 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:55.124545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 623/1255] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:41:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:41:55.130721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394912-42a0-4e82-8f8f-920c3387f656 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:55.235610 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394912-42a0-4e82-8f8f-920c3387f656 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:55.269981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dbbd6-8705-4597-b6c0-0bb13f6baa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:55.270855 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dbbd6-8705-4597-b6c0-0bb13f6baa23 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:55.271136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dbbd6-8705-4597-b6c0-0bb13f6baa23 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:55.271408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dbbd6-8705-4597-b6c0-0bb13f6baa23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:55.282040 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dbbd6-8705-4597-b6c0-0bb13f6baa23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:55.283537 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dbbd6-8705-4597-b6c0-0bb13f6baa23 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-971dbbd6-8705-4597-b6c0-0bb13f6baa23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:41:55.301917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:55.302923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:55.304599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:41:55.304599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:41:55.304599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:55.304599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:41:55.304599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 633/1256] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:55.309546 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719bf0f-f8b1-4918-b238-950b01f6cca2 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:55.409844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719bf0f-f8b1-4918-b238-950b01f6cca2 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:55.442975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a23e4-c016-4c06-bbf1-8f039306aec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:55.443630 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a23e4-c016-4c06-bbf1-8f039306aec3 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:55.443912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a23e4-c016-4c06-bbf1-8f039306aec3 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:55.444188 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a23e4-c016-4c06-bbf1-8f039306aec3 None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:55.444909 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a23e4-c016-4c06-bbf1-8f039306aec3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:55.445212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a23e4-c016-4c06-bbf1-8f039306aec3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:55.446568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a23e4-c016-4c06-bbf1-8f039306aec3 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:55.457700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 624/1257] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:55.463438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dbbd6-8705-4597-b6c0-0bb13f6baa23 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:55.565015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dbbd6-8705-4597-b6c0-0bb13f6baa23 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:55.605084 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa392371-4c18-4e58-9099-9ea3df27c911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:55.606045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa392371-4c18-4e58-9099-9ea3df27c911 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:55.606412 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa392371-4c18-4e58-9099-9ea3df27c911 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:55.606694 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa392371-4c18-4e58-9099-9ea3df27c911 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:55.614925 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa392371-4c18-4e58-9099-9ea3df27c911 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:55.616702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa392371-4c18-4e58-9099-9ea3df27c911 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-fa392371-4c18-4e58-9099-9ea3df27c911 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:41:55.633627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:55.635050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:55.635940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:41:55.635940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:41:55.635940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:41:55.635940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:41:55.635940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:41:55.635940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:41:55.635940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:41:55.635940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:41:55.635940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 634/1258] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:41:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:55.641516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a23e4-c016-4c06-bbf1-8f039306aec3 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:55.736380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a23e4-c016-4c06-bbf1-8f039306aec3 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:55.767512 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742ca65-0c74-47cc-8a3d-2e48d2efb673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:55.768150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742ca65-0c74-47cc-8a3d-2e48d2efb673 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:55.768432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742ca65-0c74-47cc-8a3d-2e48d2efb673 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:55.768706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742ca65-0c74-47cc-8a3d-2e48d2efb673 None admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:55.769478 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742ca65-0c74-47cc-8a3d-2e48d2efb673 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:55.769782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742ca65-0c74-47cc-8a3d-2e48d2efb673 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:55.771250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742ca65-0c74-47cc-8a3d-2e48d2efb673 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:55.783294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 625/1259] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:41:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:55.789756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa392371-4c18-4e58-9099-9ea3df27c911 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:55.881666 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa392371-4c18-4e58-9099-9ea3df27c911 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:55.912454 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3432b82-dbeb-4e35-ad5f-82fa17c3e67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:55.913077 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3432b82-dbeb-4e35-ad5f-82fa17c3e67c None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:55.913896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3432b82-dbeb-4e35-ad5f-82fa17c3e67c None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:55.914180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3432b82-dbeb-4e35-ad5f-82fa17c3e67c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:55.915013 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3432b82-dbeb-4e35-ad5f-82fa17c3e67c None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=8124ab8abcc640c5a48b3e0216afa48d) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:55.915322 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3432b82-dbeb-4e35-ad5f-82fa17c3e67c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:55.916738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3432b82-dbeb-4e35-ad5f-82fa17c3e67c None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:55.921864 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 635/1260] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:41:55 2022] GET /identity/v3/role_assignments?user.id=8124ab8abcc640c5a48b3e0216afa48d&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 259 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:41:58.088720 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0fb45-d2a5-497c-89aa-e100d7a4660f None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:58.089288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0fb45-d2a5-497c-89aa-e100d7a4660f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:58.089288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0fb45-d2a5-497c-89aa-e100d7a4660f None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:58.090635 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 626/1261] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:41:58.095614 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d655a-0e9c-4413-9c78-bba1be69b7d7 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:58.096019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d655a-0e9c-4413-9c78-bba1be69b7d7 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:58.096165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d655a-0e9c-4413-9c78-bba1be69b7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:58.112686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a32d655a-0e9c-4413-9c78-bba1be69b7d7 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:41:58.113847 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a32d655a-0e9c-4413-9c78-bba1be69b7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:41:58.176007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 636/1262] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:41:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:41:58.199051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0fb45-d2a5-497c-89aa-e100d7a4660f None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:58.294030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0fb45-d2a5-497c-89aa-e100d7a4660f None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:58.325532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf3a2e-4de2-472d-80ef-d226b1fdd65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:58.326300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3a2e-4de2-472d-80ef-d226b1fdd65d None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:58.326583 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3a2e-4de2-472d-80ef-d226b1fdd65d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:58.326889 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3a2e-4de2-472d-80ef-d226b1fdd65d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:58.336380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf3a2e-4de2-472d-80ef-d226b1fdd65d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-94cf3a2e-4de2-472d-80ef-d226b1fdd65d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:41:58.343847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:58.344934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:58.346164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:58.346164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:58.346164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:58.346164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:58.346164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:41:58.346164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:41:58.346164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:41:58.346164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:41:58.346164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:41:58.346164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:41:58.346164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:41:58.346164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:41:58.346164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 627/1263] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:41:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:58.351170 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d655a-0e9c-4413-9c78-bba1be69b7d7 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:58.444185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d655a-0e9c-4413-9c78-bba1be69b7d7 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:58.475142 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c00f86-a6e4-4cfe-9406-b7a7be11c18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:58.476309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c00f86-a6e4-4cfe-9406-b7a7be11c18b None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:58.476587 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c00f86-a6e4-4cfe-9406-b7a7be11c18b None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:58.476857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c00f86-a6e4-4cfe-9406-b7a7be11c18b None admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:58.477583 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c00f86-a6e4-4cfe-9406-b7a7be11c18b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:58.477884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c00f86-a6e4-4cfe-9406-b7a7be11c18b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:58.479284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c00f86-a6e4-4cfe-9406-b7a7be11c18b None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:58.486850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 637/1264] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:41:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:58.493105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf3a2e-4de2-472d-80ef-d226b1fdd65d None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:58.591239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf3a2e-4de2-472d-80ef-d226b1fdd65d None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:58.628339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24cddc-222f-4a74-b2ee-01671546a0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:58.629123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24cddc-222f-4a74-b2ee-01671546a0dd None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:58.629487 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24cddc-222f-4a74-b2ee-01671546a0dd None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:58.629763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24cddc-222f-4a74-b2ee-01671546a0dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:58.641920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24cddc-222f-4a74-b2ee-01671546a0dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:58.643506 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24cddc-222f-4a74-b2ee-01671546a0dd None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-3c24cddc-222f-4a74-b2ee-01671546a0dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:41:58.660687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:58.661718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:58.662832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:41:58.662832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:41:58.662832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:58.662832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:41:58.662832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 628/1265] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:58.668156 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c00f86-a6e4-4cfe-9406-b7a7be11c18b None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:58.761007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c00f86-a6e4-4cfe-9406-b7a7be11c18b None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:58.797954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6103ae-ef30-45dd-acc7-b600974a9c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:58.798599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6103ae-ef30-45dd-acc7-b600974a9c3f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:58.798909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6103ae-ef30-45dd-acc7-b600974a9c3f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:58.799185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6103ae-ef30-45dd-acc7-b600974a9c3f None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:58.799889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6103ae-ef30-45dd-acc7-b600974a9c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:58.800190 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6103ae-ef30-45dd-acc7-b600974a9c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:58.801540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6103ae-ef30-45dd-acc7-b600974a9c3f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:58.812492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 638/1266] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:41:58.818066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24cddc-222f-4a74-b2ee-01671546a0dd None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:58.912096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24cddc-222f-4a74-b2ee-01671546a0dd None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:58.943463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a413d-bec5-495e-8254-9a7e6d59e098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:58.944236 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a413d-bec5-495e-8254-9a7e6d59e098 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:58.945033 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a413d-bec5-495e-8254-9a7e6d59e098 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:58.945309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a413d-bec5-495e-8254-9a7e6d59e098 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:58.952393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a413d-bec5-495e-8254-9a7e6d59e098 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:58.953826 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a413d-bec5-495e-8254-9a7e6d59e098 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-d08a413d-bec5-495e-8254-9a7e6d59e098 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:41:58.968074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:58.969154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:58.970475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:58.970475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:58.970475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:41:58.970475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:41:58.970475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:41:58.970475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:41:58.970475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:41:58.970475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:41:58.970475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 21:41:58.970475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:41:58.970475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 629/1267] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:58.974747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6103ae-ef30-45dd-acc7-b600974a9c3f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:59.073879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6103ae-ef30-45dd-acc7-b600974a9c3f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:59.106480 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca5d3e-4ca4-4492-8fbd-f12470086d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:59.107164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca5d3e-4ca4-4492-8fbd-f12470086d16 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:59.107445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca5d3e-4ca4-4492-8fbd-f12470086d16 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:59.107720 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca5d3e-4ca4-4492-8fbd-f12470086d16 None admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:59.108536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca5d3e-4ca4-4492-8fbd-f12470086d16 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:59.108843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca5d3e-4ca4-4492-8fbd-f12470086d16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:59.110265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca5d3e-4ca4-4492-8fbd-f12470086d16 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:59.126514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 639/1268] 10.222.0.35 () {56 vars in 1143 bytes} [Fri May 13 21:41:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:59.132572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a413d-bec5-495e-8254-9a7e6d59e098 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:59.226432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a413d-bec5-495e-8254-9a7e6d59e098 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:59.257847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e544b5-cce2-4b17-8995-d48c469274d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:59.258622 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e544b5-cce2-4b17-8995-d48c469274d6 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:59.258928 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e544b5-cce2-4b17-8995-d48c469274d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:59.259206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e544b5-cce2-4b17-8995-d48c469274d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:59.269273 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e544b5-cce2-4b17-8995-d48c469274d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:59.270765 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e544b5-cce2-4b17-8995-d48c469274d6 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-c7e544b5-cce2-4b17-8995-d48c469274d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:41:59.287450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:59.289003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:59.290103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:41:59.290103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:41:59.290103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:41:59.290103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:41:59.290103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 630/1269] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:59.294697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca5d3e-4ca4-4492-8fbd-f12470086d16 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:59.391571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca5d3e-4ca4-4492-8fbd-f12470086d16 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:59.426058 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2913dce-cac2-4a62-b28e-769bb576b79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:59.426736 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2913dce-cac2-4a62-b28e-769bb576b79f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:59.427014 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2913dce-cac2-4a62-b28e-769bb576b79f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:59.427290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2913dce-cac2-4a62-b28e-769bb576b79f None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:59.427994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2913dce-cac2-4a62-b28e-769bb576b79f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:59.428299 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2913dce-cac2-4a62-b28e-769bb576b79f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:59.429698 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2913dce-cac2-4a62-b28e-769bb576b79f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:59.440421 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 640/1270] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:41:59.446019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e544b5-cce2-4b17-8995-d48c469274d6 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:59.541374 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e544b5-cce2-4b17-8995-d48c469274d6 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:59.573926 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56024cbe-2f0b-4ffd-b923-4c9ca3a2796b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:59.574931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56024cbe-2f0b-4ffd-b923-4c9ca3a2796b None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:59.575203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56024cbe-2f0b-4ffd-b923-4c9ca3a2796b None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:59.575483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56024cbe-2f0b-4ffd-b923-4c9ca3a2796b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:59.583855 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56024cbe-2f0b-4ffd-b923-4c9ca3a2796b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:59.586173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56024cbe-2f0b-4ffd-b923-4c9ca3a2796b None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-56024cbe-2f0b-4ffd-b923-4c9ca3a2796b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:41:59.604357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:41:59.605621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:41:59.606547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:41:59.606547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:41:59.606547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:41:59.606547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:41:59.606547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:41:59.606547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:41:59.606547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:41:59.606547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:41:59.606547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 631/1271] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:41:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:41:59.611892 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2913dce-cac2-4a62-b28e-769bb576b79f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:59.707553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2913dce-cac2-4a62-b28e-769bb576b79f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:59.738915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d490357-b56e-42aa-9162-38867bbd4093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:59.739552 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490357-b56e-42aa-9162-38867bbd4093 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:59.739832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490357-b56e-42aa-9162-38867bbd4093 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:59.740105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490357-b56e-42aa-9162-38867bbd4093 None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:59.740869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d490357-b56e-42aa-9162-38867bbd4093 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:41:59.741220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d490357-b56e-42aa-9162-38867bbd4093 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:59.742637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d490357-b56e-42aa-9162-38867bbd4093 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:59.754151 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 641/1272] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:41:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:41:59.760728 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56024cbe-2f0b-4ffd-b923-4c9ca3a2796b None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:41:59.854636 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56024cbe-2f0b-4ffd-b923-4c9ca3a2796b None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:41:59.886777 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274a8f7-aca6-40e9-a3de-8624cae912ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:41:59.887640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274a8f7-aca6-40e9-a3de-8624cae912ee None admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:41:59.887924 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274a8f7-aca6-40e9-a3de-8624cae912ee None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:41:59.888202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274a8f7-aca6-40e9-a3de-8624cae912ee None admin] PATH_INFO: `/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/8124ab8abcc640c5a48b3e0216afa48d/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:41:59.890120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274a8f7-aca6-40e9-a3de-8624cae912ee None admin] RBAC: Authorizing `identity:create_grant(project_id=ac834e84b2c84261ade311a6dfbbeb20, user_id=8124ab8abcc640c5a48b3e0216afa48d, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:41:59.891582 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274a8f7-aca6-40e9-a3de-8624cae912ee None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:41:59.904228 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 632/1273] 10.222.0.35 () {58 vars in 1439 bytes} [Fri May 13 21:41:59 2022] PUT /identity/v3/projects/ac834e84b2c84261ade311a6dfbbeb20/users/8124ab8abcc640c5a48b3e0216afa48d/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:42:02.058943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968286a-9c40-4453-a528-5e006b109569 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:02.059509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968286a-9c40-4453-a528-5e006b109569 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:02.059509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968286a-9c40-4453-a528-5e006b109569 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:02.060886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 642/1274] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:42:02.066054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1bd6b-94c3-4130-8a4d-41fe1ac35331 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:02.066446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1bd6b-94c3-4130-8a4d-41fe1ac35331 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:02.066601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1bd6b-94c3-4130-8a4d-41fe1ac35331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:02.083289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-39f1bd6b-94c3-4130-8a4d-41fe1ac35331 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:42:02.084463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-39f1bd6b-94c3-4130-8a4d-41fe1ac35331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:42:02.146958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 633/1275] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:42:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:42:02.170871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968286a-9c40-4453-a528-5e006b109569 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:02.265307 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968286a-9c40-4453-a528-5e006b109569 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:02.296462 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf6bfc-aac5-4524-a3b7-43fe631120ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:02.297225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf6bfc-aac5-4524-a3b7-43fe631120ae None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:02.297504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf6bfc-aac5-4524-a3b7-43fe631120ae None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:02.297775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf6bfc-aac5-4524-a3b7-43fe631120ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:02.307726 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf6bfc-aac5-4524-a3b7-43fe631120ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-0caf6bfc-aac5-4524-a3b7-43fe631120ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 21:42:02.315202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:42:02.316332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:42:02.317338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:42:02.317338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:42:02.317338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:42:02.317338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:42:02.317338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 21:42:02.317338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 21:42:02.317338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 21:42:02.317338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 21:42:02.317338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 21:42:02.317338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 21:42:02.317338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 21:42:02.317338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:42:02.317338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 643/1276] 10.222.0.35 () {56 vars in 1096 bytes} [Fri May 13 21:42:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:42:02.322820 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1bd6b-94c3-4130-8a4d-41fe1ac35331 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:02.414607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1bd6b-94c3-4130-8a4d-41fe1ac35331 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:02.447021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05640c-1897-4f99-a63a-2fd6c747853a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:02.447677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05640c-1897-4f99-a63a-2fd6c747853a None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:02.447961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05640c-1897-4f99-a63a-2fd6c747853a None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:02.448238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05640c-1897-4f99-a63a-2fd6c747853a None admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:02.448973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05640c-1897-4f99-a63a-2fd6c747853a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:42:02.449276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05640c-1897-4f99-a63a-2fd6c747853a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:02.450636 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05640c-1897-4f99-a63a-2fd6c747853a None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:02.458297 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 634/1277] 10.222.0.35 () {56 vars in 1099 bytes} [Fri May 13 21:42:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:42:02.463857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf6bfc-aac5-4524-a3b7-43fe631120ae None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:02.556426 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf6bfc-aac5-4524-a3b7-43fe631120ae None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:02.595577 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aec771-0bcf-4f1f-ae27-26b35ecc7288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:02.596501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aec771-0bcf-4f1f-ae27-26b35ecc7288 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:02.596940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aec771-0bcf-4f1f-ae27-26b35ecc7288 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:02.597327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aec771-0bcf-4f1f-ae27-26b35ecc7288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:02.609752 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aec771-0bcf-4f1f-ae27-26b35ecc7288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:02.611690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aec771-0bcf-4f1f-ae27-26b35ecc7288 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-38aec771-0bcf-4f1f-ae27-26b35ecc7288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:42:02.633914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:42:02.634910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:42:02.635806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:42:02.635806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:42:02.635806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:42:02.635806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:42:02.635806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 644/1278] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:42:02.642246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05640c-1897-4f99-a63a-2fd6c747853a None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:02.736769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05640c-1897-4f99-a63a-2fd6c747853a None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:02.768319 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25e4be-1535-4913-b3e4-8138846c7f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:02.768958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25e4be-1535-4913-b3e4-8138846c7f9e None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:02.769240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25e4be-1535-4913-b3e4-8138846c7f9e None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:02.769512 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25e4be-1535-4913-b3e4-8138846c7f9e None admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:02.770217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25e4be-1535-4913-b3e4-8138846c7f9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:42:02.770519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25e4be-1535-4913-b3e4-8138846c7f9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:02.771907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25e4be-1535-4913-b3e4-8138846c7f9e None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:02.783095 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 635/1279] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:42:02.788675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aec771-0bcf-4f1f-ae27-26b35ecc7288 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:02.883262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aec771-0bcf-4f1f-ae27-26b35ecc7288 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:02.913626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad939a9-4b01-4e96-bb7a-f35daa298f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:02.914957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad939a9-4b01-4e96-bb7a-f35daa298f28 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:02.915243 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad939a9-4b01-4e96-bb7a-f35daa298f28 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:02.915518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad939a9-4b01-4e96-bb7a-f35daa298f28 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:02.922482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad939a9-4b01-4e96-bb7a-f35daa298f28 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:02.923971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad939a9-4b01-4e96-bb7a-f35daa298f28 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-cad939a9-4b01-4e96-bb7a-f35daa298f28 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 21:42:02.938183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:42:02.939368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:42:02.940693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:42:02.940693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:42:02.940693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 21:42:02.940693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 21:42:02.940693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 21:42:02.940693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 21:42:02.940693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 21:42:02.940693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 21:42:02.940693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 21:42:02.940693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:42:02.940693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 645/1280] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:42:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:42:02.945015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25e4be-1535-4913-b3e4-8138846c7f9e None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:03.037848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25e4be-1535-4913-b3e4-8138846c7f9e None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:03.069192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb83a4-9e42-4b63-bc89-737d99fde67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:03.069833 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb83a4-9e42-4b63-bc89-737d99fde67d None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:03.070132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb83a4-9e42-4b63-bc89-737d99fde67d None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:03.070405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb83a4-9e42-4b63-bc89-737d99fde67d None admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:03.071249 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb83a4-9e42-4b63-bc89-737d99fde67d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:42:03.071553 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb83a4-9e42-4b63-bc89-737d99fde67d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:03.072968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb83a4-9e42-4b63-bc89-737d99fde67d None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:03.089365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 636/1281] 10.222.0.35 () {56 vars in 1143 bytes} [Fri May 13 21:42:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:42:03.094818 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad939a9-4b01-4e96-bb7a-f35daa298f28 None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:03.188522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad939a9-4b01-4e96-bb7a-f35daa298f28 None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:03.219683 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a5718-ea31-412c-b221-605df4b71d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:03.220315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a5718-ea31-412c-b221-605df4b71d2f None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:03.220594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a5718-ea31-412c-b221-605df4b71d2f None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:03.220869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a5718-ea31-412c-b221-605df4b71d2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:03.282911 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a5718-ea31-412c-b221-605df4b71d2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:03.284352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a5718-ea31-412c-b221-605df4b71d2f None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:03.374502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 646/1282] 10.222.0.35 () {58 vars in 1282 bytes} [Fri May 13 21:42:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 21:42:03.380145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb83a4-9e42-4b63-bc89-737d99fde67d None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:03.475880 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb83a4-9e42-4b63-bc89-737d99fde67d None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:03.506836 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c782a4-8f48-440f-b8c6-cb1758cd8486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:03.508184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c782a4-8f48-440f-b8c6-cb1758cd8486 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:03.508482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c782a4-8f48-440f-b8c6-cb1758cd8486 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:03.508753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c782a4-8f48-440f-b8c6-cb1758cd8486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:03.518210 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c782a4-8f48-440f-b8c6-cb1758cd8486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:03.519680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c782a4-8f48-440f-b8c6-cb1758cd8486 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-46c782a4-8f48-440f-b8c6-cb1758cd8486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 21:42:03.537453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:42:03.538494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:42:03.539662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 21:42:03.539662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 21:42:03.539662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 21:42:03.539662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:42:03.539662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 637/1283] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:42:03.544287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a5718-ea31-412c-b221-605df4b71d2f None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:03.639075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a5718-ea31-412c-b221-605df4b71d2f None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:03.670689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226078b-a98a-45bd-a375-39644231ad4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:03.671358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226078b-a98a-45bd-a375-39644231ad4a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:03.671637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226078b-a98a-45bd-a375-39644231ad4a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:03.671909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226078b-a98a-45bd-a375-39644231ad4a None admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:03.672610 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226078b-a98a-45bd-a375-39644231ad4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:42:03.672910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226078b-a98a-45bd-a375-39644231ad4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:03.674267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226078b-a98a-45bd-a375-39644231ad4a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:03.684838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 647/1284] 10.222.0.35 () {56 vars in 1109 bytes} [Fri May 13 21:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:42:03.690357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c782a4-8f48-440f-b8c6-cb1758cd8486 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:03.784050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c782a4-8f48-440f-b8c6-cb1758cd8486 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:03.815440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835910de-0782-4459-969c-8637851d6af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:03.816205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835910de-0782-4459-969c-8637851d6af7 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:03.816484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835910de-0782-4459-969c-8637851d6af7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:03.816753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835910de-0782-4459-969c-8637851d6af7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:03.823684 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835910de-0782-4459-969c-8637851d6af7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:03.825114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835910de-0782-4459-969c-8637851d6af7 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application [None req-835910de-0782-4459-969c-8637851d6af7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 21:42:03.838376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application with Lock( May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._enter() May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.creator() May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:42:03.839527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application created_value = creator( May 13 21:42:03.845298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 21:42:03.845298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 21:42:03.845298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 21:42:03.845298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 21:42:03.845298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 21:42:03.845298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 21:42:03.845298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 21:42:03.845298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: ERROR keystone.server.flask.application May 13 21:42:03.845298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 638/1285] 10.222.0.35 () {56 vars in 1111 bytes} [Fri May 13 21:42:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:42:03.845894 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226078b-a98a-45bd-a375-39644231ad4a None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:03.939106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226078b-a98a-45bd-a375-39644231ad4a None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:03.970717 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823ad40-cf65-4f4d-ac3c-35cad76ef1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:03.971364 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823ad40-cf65-4f4d-ac3c-35cad76ef1a1 None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:03.971645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823ad40-cf65-4f4d-ac3c-35cad76ef1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:03.971921 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823ad40-cf65-4f4d-ac3c-35cad76ef1a1 None admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:03.972690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823ad40-cf65-4f4d-ac3c-35cad76ef1a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:42:03.973002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823ad40-cf65-4f4d-ac3c-35cad76ef1a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:03.974432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823ad40-cf65-4f4d-ac3c-35cad76ef1a1 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:03.986443 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 648/1286] 10.222.0.35 () {56 vars in 1148 bytes} [Fri May 13 21:42:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:42:03.992743 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835910de-0782-4459-969c-8637851d6af7 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:04.089358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835910de-0782-4459-969c-8637851d6af7 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:04.120174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203fd64-d9c1-4bef-8604-74534dd477e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:04.120806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203fd64-d9c1-4bef-8604-74534dd477e6 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:04.121087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203fd64-d9c1-4bef-8604-74534dd477e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:04.121365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203fd64-d9c1-4bef-8604-74534dd477e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:04.122179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203fd64-d9c1-4bef-8604-74534dd477e6 None admin] RBAC: Adding query filter params (role.id=617e4d9732cf429b9522fc0b680ef0be, scope.project.id=ac834e84b2c84261ade311a6dfbbeb20, user.id=8124ab8abcc640c5a48b3e0216afa48d) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:42:04.122485 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203fd64-d9c1-4bef-8604-74534dd477e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:04.124187 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203fd64-d9c1-4bef-8604-74534dd477e6 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:04.129999 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 639/1287] 10.222.0.35 () {56 vars in 1374 bytes} [Fri May 13 21:42:03 2022] GET /identity/v3/role_assignments?user.id=8124ab8abcc640c5a48b3e0216afa48d&scope.project.id=ac834e84b2c84261ade311a6dfbbeb20&role.id=617e4d9732cf429b9522fc0b680ef0be => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:42:06.333490 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b34e9f-fd07-4c89-83bd-eeefbcc1e2c8 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:06.334028 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b34e9f-fd07-4c89-83bd-eeefbcc1e2c8 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:06.334028 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b34e9f-fd07-4c89-83bd-eeefbcc1e2c8 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:06.335434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 649/1288] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:42:06.340605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251ea9f-27d3-42b3-a4a0-5386ee46c61d None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:06.341047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251ea9f-27d3-42b3-a4a0-5386ee46c61d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:06.341164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251ea9f-27d3-42b3-a4a0-5386ee46c61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:06.357301 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-8251ea9f-27d3-42b3-a4a0-5386ee46c61d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:42:06.358473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8251ea9f-27d3-42b3-a4a0-5386ee46c61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:42:06.425626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 640/1289] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:42:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:42:06.448351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b34e9f-fd07-4c89-83bd-eeefbcc1e2c8 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:06.542229 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b34e9f-fd07-4c89-83bd-eeefbcc1e2c8 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:06.575865 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2c976-f150-4d38-a801-448811387b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:06.576585 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2c976-f150-4d38-a801-448811387b2a None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:06.576871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2c976-f150-4d38-a801-448811387b2a None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:06.577194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2c976-f150-4d38-a801-448811387b2a None admin] PATH_INFO: `/v3/services/placement` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:06.577800 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2c976-f150-4d38-a801-448811387b2a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:06.579285 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2c976-f150-4d38-a801-448811387b2a None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-47b2c976-f150-4d38-a801-448811387b2a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:42:06.592038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:42:06.593501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:42:06.594625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 650/1290] 10.222.0.35 () {56 vars in 1117 bytes} [Fri May 13 21:42:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:42:06.599842 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251ea9f-27d3-42b3-a4a0-5386ee46c61d None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:06.705210 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251ea9f-27d3-42b3-a4a0-5386ee46c61d None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:06.736709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d624f-bc85-4213-bd5f-8b56337c2706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:06.737362 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d624f-bc85-4213-bd5f-8b56337c2706 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:06.737644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d624f-bc85-4213-bd5f-8b56337c2706 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:06.737917 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d624f-bc85-4213-bd5f-8b56337c2706 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:06.738611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d624f-bc85-4213-bd5f-8b56337c2706 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:42:06.738944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d624f-bc85-4213-bd5f-8b56337c2706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:06.740306 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d624f-bc85-4213-bd5f-8b56337c2706 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:06.745512 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 641/1291] 10.222.0.35 () {56 vars in 1116 bytes} [Fri May 13 21:42:06 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:42:06.751498 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2c976-f150-4d38-a801-448811387b2a None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:06.843896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2c976-f150-4d38-a801-448811387b2a None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:06.875463 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0615a7-1501-42bb-84cc-31a0839683ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:06.876090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0615a7-1501-42bb-84cc-31a0839683ee None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:06.876370 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0615a7-1501-42bb-84cc-31a0839683ee None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:06.876639 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0615a7-1501-42bb-84cc-31a0839683ee None admin] PATH_INFO: `/v3/services` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:06.877314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0615a7-1501-42bb-84cc-31a0839683ee None admin] RBAC: Adding query filter params (type=placement) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:42:06.877614 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0615a7-1501-42bb-84cc-31a0839683ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:06.878987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0615a7-1501-42bb-84cc-31a0839683ee None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:06.883220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 651/1292] 10.222.0.35 () {56 vars in 1116 bytes} [Fri May 13 21:42:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:42:09.041449 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b401bb-4cc4-459b-bd53-d69e41526bc5 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:09.041990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b401bb-4cc4-459b-bd53-d69e41526bc5 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:09.041990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b401bb-4cc4-459b-bd53-d69e41526bc5 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:09.043384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 642/1293] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:42:09.048363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706d16b-928b-40aa-a74f-5720af26c682 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:09.048756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706d16b-928b-40aa-a74f-5720af26c682 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:09.048915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706d16b-928b-40aa-a74f-5720af26c682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:09.064956 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-0706d16b-928b-40aa-a74f-5720af26c682 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:42:09.066116 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-0706d16b-928b-40aa-a74f-5720af26c682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:42:09.127427 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 652/1294] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:42:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:42:09.149767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b401bb-4cc4-459b-bd53-d69e41526bc5 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:09.244629 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b401bb-4cc4-459b-bd53-d69e41526bc5 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:09.276144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba383d-33a9-4976-a17d-e3a885dab2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:09.276783 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba383d-33a9-4976-a17d-e3a885dab2f4 None admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:09.277062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba383d-33a9-4976-a17d-e3a885dab2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:09.277338 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba383d-33a9-4976-a17d-e3a885dab2f4 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:09.278039 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba383d-33a9-4976-a17d-e3a885dab2f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:09.279429 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba383d-33a9-4976-a17d-e3a885dab2f4 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:09.292302 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 643/1295] 10.222.0.35 () {60 vars in 1141 bytes} [Fri May 13 21:42:09 2022] POST /identity/v3/services => generated 246 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:42:11.473159 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc0356-7c5e-4757-91b1-70397167ee4a None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:11.473692 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc0356-7c5e-4757-91b1-70397167ee4a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:11.473692 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc0356-7c5e-4757-91b1-70397167ee4a None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:11.475121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 653/1296] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:42:11.480217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a084511-cd54-4fe7-aa93-b7aaaa7ea130 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:11.480593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a084511-cd54-4fe7-aa93-b7aaaa7ea130 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:11.480776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a084511-cd54-4fe7-aa93-b7aaaa7ea130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:11.497292 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-7a084511-cd54-4fe7-aa93-b7aaaa7ea130 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:42:11.498725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-7a084511-cd54-4fe7-aa93-b7aaaa7ea130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:42:11.567302 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 644/1297] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:42:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 21:42:11.590900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc0356-7c5e-4757-91b1-70397167ee4a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:11.687380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc0356-7c5e-4757-91b1-70397167ee4a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:11.718518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb8e98-ef8b-424d-91e5-12402931fccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:11.719321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb8e98-ef8b-424d-91e5-12402931fccd None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:11.719603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb8e98-ef8b-424d-91e5-12402931fccd None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:11.719876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb8e98-ef8b-424d-91e5-12402931fccd None admin] PATH_INFO: `/v3/services/placement` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:11.720514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb8e98-ef8b-424d-91e5-12402931fccd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:11.721878 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb8e98-ef8b-424d-91e5-12402931fccd None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-b5bb8e98-ef8b-424d-91e5-12402931fccd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:42:11.733859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:42:11.735030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:42:11.735904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 654/1298] 10.222.0.35 () {56 vars in 1117 bytes} [Fri May 13 21:42:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:42:11.741008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a084511-cd54-4fe7-aa93-b7aaaa7ea130 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:11.842263 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a084511-cd54-4fe7-aa93-b7aaaa7ea130 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:11.874612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131e832-d71f-4cd2-850d-db52860055e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:11.875292 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131e832-d71f-4cd2-850d-db52860055e7 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:11.875576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131e832-d71f-4cd2-850d-db52860055e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:11.875850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131e832-d71f-4cd2-850d-db52860055e7 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:11.876546 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131e832-d71f-4cd2-850d-db52860055e7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:42:11.876849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131e832-d71f-4cd2-850d-db52860055e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:11.878234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131e832-d71f-4cd2-850d-db52860055e7 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:11.883717 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 645/1299] 10.222.0.35 () {56 vars in 1116 bytes} [Fri May 13 21:42:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:42:11.889571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb8e98-ef8b-424d-91e5-12402931fccd None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:11.982841 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb8e98-ef8b-424d-91e5-12402931fccd None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:12.013940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f03f6c-2827-4d79-82ca-ac659a94b7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:12.014509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f03f6c-2827-4d79-82ca-ac659a94b7ed None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:12.014813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f03f6c-2827-4d79-82ca-ac659a94b7ed None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:12.015087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f03f6c-2827-4d79-82ca-ac659a94b7ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:12.015853 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f03f6c-2827-4d79-82ca-ac659a94b7ed None admin] RBAC: Adding query filter params (interface=public, service_id=baca08cc3e08438da08b9075f47a7c43, region_id=RegionOne) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:42:12.016158 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f03f6c-2827-4d79-82ca-ac659a94b7ed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:12.017634 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f03f6c-2827-4d79-82ca-ac659a94b7ed None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:12.022337 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 655/1300] 10.222.0.35 () {56 vars in 1251 bytes} [Fri May 13 21:42:11 2022] GET /identity/v3/endpoints?service_id=baca08cc3e08438da08b9075f47a7c43&interface=public®ion_id=RegionOne => generated 194 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:42:12.027862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131e832-d71f-4cd2-850d-db52860055e7 None admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:12.121531 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131e832-d71f-4cd2-850d-db52860055e7 None admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:12.154023 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7773c3-65b2-4b36-acd4-1d342cbaeee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:12.154204 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7773c3-65b2-4b36-acd4-1d342cbaeee8 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:12.154415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7773c3-65b2-4b36-acd4-1d342cbaeee8 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:12.154687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7773c3-65b2-4b36-acd4-1d342cbaeee8 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:12.155381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7773c3-65b2-4b36-acd4-1d342cbaeee8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:12.156738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7773c3-65b2-4b36-acd4-1d342cbaeee8 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:12.163382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 646/1301] 10.222.0.35 () {56 vars in 1087 bytes} [Fri May 13 21:42:12 2022] GET /identity/v3/services => generated 2191 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:42:14.332490 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99fb98-41eb-49d8-b2ea-f484b47e6d73 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:14.333013 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99fb98-41eb-49d8-b2ea-f484b47e6d73 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:14.333103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99fb98-41eb-49d8-b2ea-f484b47e6d73 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:14.334416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 656/1302] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:42:14.340098 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff46af3-b54e-4a2b-856f-b656bd3fad1c None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:14.340517 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff46af3-b54e-4a2b-856f-b656bd3fad1c None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:14.340653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff46af3-b54e-4a2b-856f-b656bd3fad1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:14.357633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5ff46af3-b54e-4a2b-856f-b656bd3fad1c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:42:14.358848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5ff46af3-b54e-4a2b-856f-b656bd3fad1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:42:14.426476 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 647/1303] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:42:14 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 21:42:14.452689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99fb98-41eb-49d8-b2ea-f484b47e6d73 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:14.548103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99fb98-41eb-49d8-b2ea-f484b47e6d73 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:14.581243 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629be828-9fd7-4d7f-a57e-2a4106bdfcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:14.582076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629be828-9fd7-4d7f-a57e-2a4106bdfcbb None admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:14.582391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629be828-9fd7-4d7f-a57e-2a4106bdfcbb None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:14.582789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629be828-9fd7-4d7f-a57e-2a4106bdfcbb None admin] PATH_INFO: `/v3/services/placement` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:14.583603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629be828-9fd7-4d7f-a57e-2a4106bdfcbb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:14.585052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629be828-9fd7-4d7f-a57e-2a4106bdfcbb None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application [None req-629be828-9fd7-4d7f-a57e-2a4106bdfcbb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 21:42:14.598329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 21:42:14.599447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.get_or_create( May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application with Lock( May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self._enter() May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application return self.creator() May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application created_value = creator( May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: ERROR keystone.server.flask.application May 13 21:42:14.600291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 657/1304] 10.222.0.35 () {56 vars in 1117 bytes} [Fri May 13 21:42:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 21:42:14.606377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff46af3-b54e-4a2b-856f-b656bd3fad1c None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:14.702793 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff46af3-b54e-4a2b-856f-b656bd3fad1c None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:14.734282 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95fc2b-2432-48ea-9edb-53e7949a8bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:14.734956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95fc2b-2432-48ea-9edb-53e7949a8bf0 None admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:14.735257 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95fc2b-2432-48ea-9edb-53e7949a8bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:14.735516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95fc2b-2432-48ea-9edb-53e7949a8bf0 None admin] PATH_INFO: `/v3/services` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:14.736209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95fc2b-2432-48ea-9edb-53e7949a8bf0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:42:14.736514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95fc2b-2432-48ea-9edb-53e7949a8bf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:14.737881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95fc2b-2432-48ea-9edb-53e7949a8bf0 None admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:14.743398 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 648/1305] 10.222.0.35 () {56 vars in 1116 bytes} [Fri May 13 21:42:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:42:14.749392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629be828-9fd7-4d7f-a57e-2a4106bdfcbb None admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:42:14.843412 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629be828-9fd7-4d7f-a57e-2a4106bdfcbb None admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:42:14.875220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c3ac8-4fd8-40c4-9868-cd99aa1956f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:42:14.875859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c3ac8-4fd8-40c4-9868-cd99aa1956f2 None admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:42:14.876140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c3ac8-4fd8-40c4-9868-cd99aa1956f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:42:14.876414 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c3ac8-4fd8-40c4-9868-cd99aa1956f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:42:14.877148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c3ac8-4fd8-40c4-9868-cd99aa1956f2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:42:14.878504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c3ac8-4fd8-40c4-9868-cd99aa1956f2 None admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:42:14.895377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 658/1306] 10.222.0.35 () {60 vars in 1144 bytes} [Fri May 13 21:42:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:43:18.010234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a17c4-e502-443d-bb23-a9fe2a4fa083 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:18.011314 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a17c4-e502-443d-bb23-a9fe2a4fa083 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:18.011817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a17c4-e502-443d-bb23-a9fe2a4fa083 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:18.017239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 649/1307] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:43:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:18.018621 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd230dc-ee6f-48b3-ab50-7ed62dbd632e None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:18.019509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd230dc-ee6f-48b3-ab50-7ed62dbd632e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:18.019998 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd230dc-ee6f-48b3-ab50-7ed62dbd632e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:18.094072 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-5cd230dc-ee6f-48b3-ab50-7ed62dbd632e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:18.095654 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-5cd230dc-ee6f-48b3-ab50-7ed62dbd632e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:18.272411 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 659/1308] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:43:18.326668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cec47a-306c-454e-b134-19cf19c1c43c None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:18.327050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cec47a-306c-454e-b134-19cf19c1c43c None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:18.327878 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cec47a-306c-454e-b134-19cf19c1c43c None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:18.331551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 650/1309] 10.222.0.35 () {54 vars in 964 bytes} [Fri May 13 21:43:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:18.337563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8a69-e5ec-4863-9002-472726cca889 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:18.337870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8a69-e5ec-4863-9002-472726cca889 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:18.338156 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8a69-e5ec-4863-9002-472726cca889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:18.463595 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-3def8a69-e5ec-4863-9002-472726cca889 None None] MFA Rules not processed for user `c4df058606c448a1bb757020e869ac33`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:18.465967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3def8a69-e5ec-4863-9002-472726cca889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:18.550286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 660/1310] 10.222.0.35 () {58 vars in 1061 bytes} [Fri May 13 21:43:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:43:18.559783 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cec47a-306c-454e-b134-19cf19c1c43c None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:43:18.561548 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-41cec47a-306c-454e-b134-19cf19c1c43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:18.687728 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cec47a-306c-454e-b134-19cf19c1c43c None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:43:18.810300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27257ad7-73d2-43b2-b688-30a36289e09a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:43:18.811413 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27257ad7-73d2-43b2-b688-30a36289e09a service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:18.811906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27257ad7-73d2-43b2-b688-30a36289e09a service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:18.812405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27257ad7-73d2-43b2-b688-30a36289e09a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:18.814277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-27257ad7-73d2-43b2-b688-30a36289e09a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:18.949357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27257ad7-73d2-43b2-b688-30a36289e09a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:43:18.950899 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27257ad7-73d2-43b2-b688-30a36289e09a service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:43:18.970817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 651/1311] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:43:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:43:21.553883 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636e24e-d53d-4488-abde-fa2fe973405b None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:21.554465 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636e24e-d53d-4488-abde-fa2fe973405b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:21.554465 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636e24e-d53d-4488-abde-fa2fe973405b None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:21.556034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 661/1312] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:43:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:21.561628 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53650a6e-b939-4d97-b178-ae7231a6425e None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:21.562138 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53650a6e-b939-4d97-b178-ae7231a6425e None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:21.562233 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53650a6e-b939-4d97-b178-ae7231a6425e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:21.652598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-53650a6e-b939-4d97-b178-ae7231a6425e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:21.653873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-53650a6e-b939-4d97-b178-ae7231a6425e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:21.755101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 652/1313] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:43:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:43:21.792659 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636e24e-d53d-4488-abde-fa2fe973405b None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:43:21.794192 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9636e24e-d53d-4488-abde-fa2fe973405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:21.895464 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636e24e-d53d-4488-abde-fa2fe973405b None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:43:21.966750 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1c976-e04a-4656-8cff-98d7c0462e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:43:21.967650 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1c976-e04a-4656-8cff-98d7c0462e41 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:21.967933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1c976-e04a-4656-8cff-98d7c0462e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:21.968211 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1c976-e04a-4656-8cff-98d7c0462e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:21.969855 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e2a1c976-e04a-4656-8cff-98d7c0462e41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:22.077636 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1c976-e04a-4656-8cff-98d7c0462e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:43:22.079298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1c976-e04a-4656-8cff-98d7c0462e41 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:43:22.098286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 662/1314] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:43:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:43:24.650069 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dcb4b-e429-4a98-8ea6-7c1682e64394 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:24.650695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dcb4b-e429-4a98-8ea6-7c1682e64394 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:24.650695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dcb4b-e429-4a98-8ea6-7c1682e64394 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:24.652021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 653/1315] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:24.657477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2524a-eaeb-4fa9-9311-8fa8835413ff None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:24.657871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2524a-eaeb-4fa9-9311-8fa8835413ff None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:24.658030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2524a-eaeb-4fa9-9311-8fa8835413ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:24.719562 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-57d2524a-eaeb-4fa9-9311-8fa8835413ff None None] MFA Rules not processed for user `c5810ecbb17a4b1ea3085dd850c6fa2d`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:24.720755 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-57d2524a-eaeb-4fa9-9311-8fa8835413ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:24.815092 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 663/1316] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:43:24 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:43:24.855154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82dcb4b-e429-4a98-8ea6-7c1682e64394 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:43:24.856689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f82dcb4b-e429-4a98-8ea6-7c1682e64394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:24.955854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82dcb4b-e429-4a98-8ea6-7c1682e64394 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:43:25.024515 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581d017-1c60-4393-9a81-4201ba14bd8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:43:25.025153 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581d017-1c60-4393-9a81-4201ba14bd8d service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:25.025436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581d017-1c60-4393-9a81-4201ba14bd8d service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:25.025710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581d017-1c60-4393-9a81-4201ba14bd8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:25.027356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-1581d017-1c60-4393-9a81-4201ba14bd8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:25.130648 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581d017-1c60-4393-9a81-4201ba14bd8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:43:25.132107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581d017-1c60-4393-9a81-4201ba14bd8d service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:43:25.151898 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 654/1317] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:43:24 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:43:27.862141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac77d23-9a7e-4e65-ba13-85b9df5b44ce None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:27.862579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac77d23-9a7e-4e65-ba13-85b9df5b44ce None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:27.862695 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac77d23-9a7e-4e65-ba13-85b9df5b44ce None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:27.864113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 664/1318] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:27.869476 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b9690-a8d5-4cd7-8a42-2fe37e10d77b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:27.869881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b9690-a8d5-4cd7-8a42-2fe37e10d77b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:27.870038 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b9690-a8d5-4cd7-8a42-2fe37e10d77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:27.932797 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-7f3b9690-a8d5-4cd7-8a42-2fe37e10d77b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:27.933992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-7f3b9690-a8d5-4cd7-8a42-2fe37e10d77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:28.025181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 655/1319] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:43:27 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:43:28.076309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a87d7c-91cc-4acf-bbb6-701cca44bd5f None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:28.076593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a87d7c-91cc-4acf-bbb6-701cca44bd5f None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:28.076870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a87d7c-91cc-4acf-bbb6-701cca44bd5f None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:28.078240 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 665/1320] 10.222.0.35 () {54 vars in 964 bytes} [Fri May 13 21:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:28.084266 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd78039-f75f-47a8-9df1-e3f64e00c7c5 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:28.084559 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd78039-f75f-47a8-9df1-e3f64e00c7c5 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:28.084839 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd78039-f75f-47a8-9df1-e3f64e00c7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:28.158971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-7fd78039-f75f-47a8-9df1-e3f64e00c7c5 None None] MFA Rules not processed for user `c4df058606c448a1bb757020e869ac33`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:28.160200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-7fd78039-f75f-47a8-9df1-e3f64e00c7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:28.234634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 656/1321] 10.222.0.35 () {58 vars in 1061 bytes} [Fri May 13 21:43:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:43:28.243619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a87d7c-91cc-4acf-bbb6-701cca44bd5f None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:43:28.245144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-61a87d7c-91cc-4acf-bbb6-701cca44bd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:28.345162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a87d7c-91cc-4acf-bbb6-701cca44bd5f None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:43:28.414581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30caea-d444-4ec5-98e2-2966d3eda6c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:43:28.415183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30caea-d444-4ec5-98e2-2966d3eda6c3 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:28.415465 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30caea-d444-4ec5-98e2-2966d3eda6c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:28.415741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30caea-d444-4ec5-98e2-2966d3eda6c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:28.417362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-2f30caea-d444-4ec5-98e2-2966d3eda6c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:28.519782 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30caea-d444-4ec5-98e2-2966d3eda6c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:43:28.521799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30caea-d444-4ec5-98e2-2966d3eda6c3 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:43:28.538872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 666/1322] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:43:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:43:31.846489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10859924-abbc-4f60-a98d-b3ccd1f6e7ca None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:31.847059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10859924-abbc-4f60-a98d-b3ccd1f6e7ca None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:31.847059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10859924-abbc-4f60-a98d-b3ccd1f6e7ca None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:31.848466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 657/1323] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:43:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:31.853809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12699b-d116-427e-8f84-67da3e0bd774 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:31.854270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12699b-d116-427e-8f84-67da3e0bd774 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:31.854439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12699b-d116-427e-8f84-67da3e0bd774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:31.917830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-fa12699b-d116-427e-8f84-67da3e0bd774 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:31.919034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-fa12699b-d116-427e-8f84-67da3e0bd774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:32.011116 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 667/1324] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:43:32.045991 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10859924-abbc-4f60-a98d-b3ccd1f6e7ca None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:43:32.047629 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-10859924-abbc-4f60-a98d-b3ccd1f6e7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:32.155692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10859924-abbc-4f60-a98d-b3ccd1f6e7ca None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:43:32.228307 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b094126-140b-48b0-adcc-34749e23e4ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:43:32.228955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b094126-140b-48b0-adcc-34749e23e4ad service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:32.229241 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b094126-140b-48b0-adcc-34749e23e4ad service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:32.229524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b094126-140b-48b0-adcc-34749e23e4ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:32.231544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-3b094126-140b-48b0-adcc-34749e23e4ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:32.336600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b094126-140b-48b0-adcc-34749e23e4ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:43:32.338063 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b094126-140b-48b0-adcc-34749e23e4ad service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:43:32.355968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 658/1325] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:43:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:43:34.637641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b77b03-e0b8-4847-9723-7a699dc065f2 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:34.638221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b77b03-e0b8-4847-9723-7a699dc065f2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:34.638221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b77b03-e0b8-4847-9723-7a699dc065f2 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:34.639668 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 668/1326] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:34.645051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d114721-893f-468d-b2ec-5d1e5bcc8be5 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:34.645496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d114721-893f-468d-b2ec-5d1e5bcc8be5 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:34.645791 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d114721-893f-468d-b2ec-5d1e5bcc8be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:34.708228 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-1d114721-893f-468d-b2ec-5d1e5bcc8be5 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:34.709435 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-1d114721-893f-468d-b2ec-5d1e5bcc8be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:34.804435 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 659/1327] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:43:34.838175 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b77b03-e0b8-4847-9723-7a699dc065f2 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:43:34.839816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-25b77b03-e0b8-4847-9723-7a699dc065f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:34.938274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b77b03-e0b8-4847-9723-7a699dc065f2 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:43:35.007555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135ec2a-7e09-4698-9d17-e70cb6c59a8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:43:35.008188 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135ec2a-7e09-4698-9d17-e70cb6c59a8b service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:35.008468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135ec2a-7e09-4698-9d17-e70cb6c59a8b service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:35.008743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135ec2a-7e09-4698-9d17-e70cb6c59a8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:35.010351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b135ec2a-7e09-4698-9d17-e70cb6c59a8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:35.112531 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b135ec2a-7e09-4698-9d17-e70cb6c59a8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:43:35.113961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b135ec2a-7e09-4698-9d17-e70cb6c59a8b service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:43:35.131672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 669/1328] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:43:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:43:37.395985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebe3db-5473-421c-b10a-de1bdac8ea9b None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:37.396542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebe3db-5473-421c-b10a-de1bdac8ea9b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:37.396542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebe3db-5473-421c-b10a-de1bdac8ea9b None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:37.397925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 660/1329] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:37.403258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e614a-f4c7-4129-88dd-fc95ee35edfb None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:37.403680 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e614a-f4c7-4129-88dd-fc95ee35edfb None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:37.403822 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e614a-f4c7-4129-88dd-fc95ee35edfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:37.466547 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d95e614a-f4c7-4129-88dd-fc95ee35edfb None None] MFA Rules not processed for user `c5810ecbb17a4b1ea3085dd850c6fa2d`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:37.467777 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d95e614a-f4c7-4129-88dd-fc95ee35edfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:37.560744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 670/1330] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:43:37 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:43:37.596890 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ebe3db-5473-421c-b10a-de1bdac8ea9b None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:43:37.599113 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-44ebe3db-5473-421c-b10a-de1bdac8ea9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:37.701582 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ebe3db-5473-421c-b10a-de1bdac8ea9b None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:43:37.773227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f6b4c-3cda-43cd-aa5e-1973d9d88c8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:43:37.773797 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f6b4c-3cda-43cd-aa5e-1973d9d88c8e service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:37.774081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f6b4c-3cda-43cd-aa5e-1973d9d88c8e service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:37.774358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f6b4c-3cda-43cd-aa5e-1973d9d88c8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:37.776024 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-fe8f6b4c-3cda-43cd-aa5e-1973d9d88c8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:37.881729 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f6b4c-3cda-43cd-aa5e-1973d9d88c8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:43:37.883218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f6b4c-3cda-43cd-aa5e-1973d9d88c8e service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:43:37.901322 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 661/1331] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:43:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:43:40.377153 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3468aae-c79f-42e0-9eb6-80831e30f68b None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:40.377674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3468aae-c79f-42e0-9eb6-80831e30f68b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:40.377848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3468aae-c79f-42e0-9eb6-80831e30f68b None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:40.379302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 671/1332] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:40.385056 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd2ddb-f824-4f53-8d1b-cf4434d4c45e None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:40.385478 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd2ddb-f824-4f53-8d1b-cf4434d4c45e None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:40.385621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd2ddb-f824-4f53-8d1b-cf4434d4c45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:40.453115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-b3fd2ddb-f824-4f53-8d1b-cf4434d4c45e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:40.454308 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b3fd2ddb-f824-4f53-8d1b-cf4434d4c45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:40.546452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 662/1333] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:43:40.582512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3468aae-c79f-42e0-9eb6-80831e30f68b None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:43:40.584704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d3468aae-c79f-42e0-9eb6-80831e30f68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:40.684619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3468aae-c79f-42e0-9eb6-80831e30f68b None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:43:40.755446 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea199674-2a55-430a-8bc9-8b6043520336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:43:40.756078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea199674-2a55-430a-8bc9-8b6043520336 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:40.756357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea199674-2a55-430a-8bc9-8b6043520336 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:40.756629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea199674-2a55-430a-8bc9-8b6043520336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:40.758242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ea199674-2a55-430a-8bc9-8b6043520336 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:40.865710 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea199674-2a55-430a-8bc9-8b6043520336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:43:40.867109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea199674-2a55-430a-8bc9-8b6043520336 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:43:40.884573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 672/1334] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:43:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:43:43.180201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5385730-29ff-431b-88b9-209be4a94067 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:43.180711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5385730-29ff-431b-88b9-209be4a94067 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:43.180836 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5385730-29ff-431b-88b9-209be4a94067 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:43.182107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 663/1335] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:43.187325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c49cb-fbc4-4c9e-857b-1d52c6c374a0 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:43.187732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c49cb-fbc4-4c9e-857b-1d52c6c374a0 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:43.187874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c49cb-fbc4-4c9e-857b-1d52c6c374a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:43.249741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d37c49cb-fbc4-4c9e-857b-1d52c6c374a0 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:43.250943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d37c49cb-fbc4-4c9e-857b-1d52c6c374a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:43.340430 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 673/1336] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:43:43.380877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5385730-29ff-431b-88b9-209be4a94067 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:43:43.382386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b5385730-29ff-431b-88b9-209be4a94067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:43.481251 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5385730-29ff-431b-88b9-209be4a94067 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:43:43.550028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d70be-a5bb-494a-9549-017e3f8dd321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:43:43.550658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d70be-a5bb-494a-9549-017e3f8dd321 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:43.550968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d70be-a5bb-494a-9549-017e3f8dd321 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:43.551773 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d70be-a5bb-494a-9549-017e3f8dd321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:43.553394 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0e4d70be-a5bb-494a-9549-017e3f8dd321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:43.656864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d70be-a5bb-494a-9549-017e3f8dd321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:43:43.658295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d70be-a5bb-494a-9549-017e3f8dd321 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:43:43.675716 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 664/1337] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:43:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:43:46.245511 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dac0d-4fbe-44f0-bdfc-7ee471ffecc3 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:46.246297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dac0d-4fbe-44f0-bdfc-7ee471ffecc3 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:46.246395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dac0d-4fbe-44f0-bdfc-7ee471ffecc3 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:46.247886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 674/1338] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:46.252929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55225740-e6eb-48ff-80f0-c081ba196ac9 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:46.253366 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55225740-e6eb-48ff-80f0-c081ba196ac9 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:46.253479 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55225740-e6eb-48ff-80f0-c081ba196ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:46.316214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-55225740-e6eb-48ff-80f0-c081ba196ac9 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:46.317472 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-55225740-e6eb-48ff-80f0-c081ba196ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:46.410438 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 665/1339] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:43:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:43:46.445615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dac0d-4fbe-44f0-bdfc-7ee471ffecc3 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:43:46.447179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-305dac0d-4fbe-44f0-bdfc-7ee471ffecc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:46.546277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dac0d-4fbe-44f0-bdfc-7ee471ffecc3 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:43:46.619030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa86cd-8bb4-4302-9d6b-ca6dac79da67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:43:46.619674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa86cd-8bb4-4302-9d6b-ca6dac79da67 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:46.619956 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa86cd-8bb4-4302-9d6b-ca6dac79da67 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:46.620230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa86cd-8bb4-4302-9d6b-ca6dac79da67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:46.622112 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-69fa86cd-8bb4-4302-9d6b-ca6dac79da67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:46.724286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa86cd-8bb4-4302-9d6b-ca6dac79da67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:43:46.725714 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa86cd-8bb4-4302-9d6b-ca6dac79da67 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:43:46.743248 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 675/1340] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:43:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:43:50.920272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e948b84-1685-43fc-a96d-a8081cb609c2 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:50.920813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e948b84-1685-43fc-a96d-a8081cb609c2 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:50.920813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e948b84-1685-43fc-a96d-a8081cb609c2 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:50.922101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 666/1341] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:50.928016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7455b15-9847-4a68-89ae-437dc5c0bd4d None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:50.928454 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7455b15-9847-4a68-89ae-437dc5c0bd4d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:50.928697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7455b15-9847-4a68-89ae-437dc5c0bd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:50.997715 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d7455b15-9847-4a68-89ae-437dc5c0bd4d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:50.998936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d7455b15-9847-4a68-89ae-437dc5c0bd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:51.100204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 676/1342] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:43:51.138063 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e948b84-1685-43fc-a96d-a8081cb609c2 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:43:51.140145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2e948b84-1685-43fc-a96d-a8081cb609c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:51.233763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e948b84-1685-43fc-a96d-a8081cb609c2 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:43:51.238221 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a3e3b-f9b3-4717-8701-3640ee97d0e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:43:51.239621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a3e3b-f9b3-4717-8701-3640ee97d0e5 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:51.240015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a3e3b-f9b3-4717-8701-3640ee97d0e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:51.240418 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a3e3b-f9b3-4717-8701-3640ee97d0e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:51.250518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a3e3b-f9b3-4717-8701-3640ee97d0e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:43:51.252116 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a3e3b-f9b3-4717-8701-3640ee97d0e5 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:43:51.261749 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 667/1343] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:43:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:43:54.085424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26932f59-7044-4927-ae9c-de6b4b37858c None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:54.085952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26932f59-7044-4927-ae9c-de6b4b37858c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:54.086062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26932f59-7044-4927-ae9c-de6b4b37858c None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:54.087567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 677/1344] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:54.092435 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c524e3-336a-47da-a434-ba0ea5fed440 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:54.092843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c524e3-336a-47da-a434-ba0ea5fed440 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:54.092990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c524e3-336a-47da-a434-ba0ea5fed440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:54.124195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-08c524e3-336a-47da-a434-ba0ea5fed440 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:54.125317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-08c524e3-336a-47da-a434-ba0ea5fed440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:54.204840 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 668/1345] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:43:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:43:54.240874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26932f59-7044-4927-ae9c-de6b4b37858c None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:43:54.251703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26932f59-7044-4927-ae9c-de6b4b37858c None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:43:54.255861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db3212-f610-441f-aae8-737eb58f7224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:43:54.256576 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db3212-f610-441f-aae8-737eb58f7224 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:54.256935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db3212-f610-441f-aae8-737eb58f7224 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:54.257281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db3212-f610-441f-aae8-737eb58f7224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:54.266086 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db3212-f610-441f-aae8-737eb58f7224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:43:54.267623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db3212-f610-441f-aae8-737eb58f7224 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:43:54.277517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 678/1346] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:43:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:43:58.471751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c14b5-f0a0-4f24-a10c-cd58db8412e3 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:58.472334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c14b5-f0a0-4f24-a10c-cd58db8412e3 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:58.472334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c14b5-f0a0-4f24-a10c-cd58db8412e3 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:58.473679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 669/1347] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:43:58.479673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc988d-77c4-49a6-9576-b7f3a1698551 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:58.480126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc988d-77c4-49a6-9576-b7f3a1698551 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:58.480270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc988d-77c4-49a6-9576-b7f3a1698551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:58.498762 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-42dc988d-77c4-49a6-9576-b7f3a1698551 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:43:58.500155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-42dc988d-77c4-49a6-9576-b7f3a1698551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:58.508621 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 679/1348] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:43:58.544912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c14b5-f0a0-4f24-a10c-cd58db8412e3 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:43:58.546593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-617c14b5-f0a0-4f24-a10c-cd58db8412e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:43:58.559620 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c14b5-f0a0-4f24-a10c-cd58db8412e3 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:43:58.563375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc3c97-f4b4-4e67-a5d6-8df79261d432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:43:58.565687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc3c97-f4b4-4e67-a5d6-8df79261d432 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:43:58.565687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc3c97-f4b4-4e67-a5d6-8df79261d432 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:43:58.565687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc3c97-f4b4-4e67-a5d6-8df79261d432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:43:58.574748 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc3c97-f4b4-4e67-a5d6-8df79261d432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:43:58.576519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc3c97-f4b4-4e67-a5d6-8df79261d432 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:43:58.586073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 670/1349] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:44:03.923328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f764d72-cb88-46f2-a575-f297668d4cf6 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:03.923795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f764d72-cb88-46f2-a575-f297668d4cf6 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:03.923879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f764d72-cb88-46f2-a575-f297668d4cf6 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:03.926064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 680/1350] 10.222.0.35 () {54 vars in 921 bytes} [Fri May 13 21:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:03.931031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c050afa-dd44-4b92-94dd-7a9ff5075aec None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:03.931452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c050afa-dd44-4b92-94dd-7a9ff5075aec None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:03.931584 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c050afa-dd44-4b92-94dd-7a9ff5075aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:04.026503 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-6c050afa-dd44-4b92-94dd-7a9ff5075aec None None] MFA Rules not processed for user `8124ab8abcc640c5a48b3e0216afa48d`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:04.027745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-6c050afa-dd44-4b92-94dd-7a9ff5075aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:04.140899 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 671/1351] 10.222.0.35 () {58 vars in 1018 bytes} [Fri May 13 21:44:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:08.261919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168259d-4488-4b2c-9f85-acbeb4245658 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:08.262459 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168259d-4488-4b2c-9f85-acbeb4245658 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:08.262896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168259d-4488-4b2c-9f85-acbeb4245658 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:08.264513 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 681/1352] 10.222.0.35 () {54 vars in 921 bytes} [Fri May 13 21:44:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:08.270106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bee723-7050-4dfc-bc51-2b19ab58db81 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:08.270531 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bee723-7050-4dfc-bc51-2b19ab58db81 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:08.270652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bee723-7050-4dfc-bc51-2b19ab58db81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:08.289170 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-84bee723-7050-4dfc-bc51-2b19ab58db81 None None] MFA Rules not processed for user `8124ab8abcc640c5a48b3e0216afa48d`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:08.290461 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-84bee723-7050-4dfc-bc51-2b19ab58db81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:08.299512 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 672/1353] 10.222.0.35 () {58 vars in 1018 bytes} [Fri May 13 21:44:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:10.015044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682dc20-f4d3-449f-943a-4a1d64ebc33a None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:10.015561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682dc20-f4d3-449f-943a-4a1d64ebc33a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:10.015645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682dc20-f4d3-449f-943a-4a1d64ebc33a None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:10.016989 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 682/1354] 10.222.0.35 () {54 vars in 921 bytes} [Fri May 13 21:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:10.021874 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57ce20-fbc2-43ca-a87c-bcd589d560e9 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:10.022287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57ce20-fbc2-43ca-a87c-bcd589d560e9 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:10.022433 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57ce20-fbc2-43ca-a87c-bcd589d560e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:10.040479 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-8a57ce20-fbc2-43ca-a87c-bcd589d560e9 None None] MFA Rules not processed for user `8124ab8abcc640c5a48b3e0216afa48d`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:10.041694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8a57ce20-fbc2-43ca-a87c-bcd589d560e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:10.050816 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 673/1355] 10.222.0.35 () {58 vars in 1018 bytes} [Fri May 13 21:44:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:44:13.898092 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebaef27-de9d-4099-b266-612dfca61a43 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:13.898630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebaef27-de9d-4099-b266-612dfca61a43 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:13.898872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebaef27-de9d-4099-b266-612dfca61a43 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:13.904224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 683/1356] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:44:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:13.905803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740f7d9-92c8-4924-b086-8943a0a617e4 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:13.906239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740f7d9-92c8-4924-b086-8943a0a617e4 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:13.906407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740f7d9-92c8-4924-b086-8943a0a617e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:13.924220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-b740f7d9-92c8-4924-b086-8943a0a617e4 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:13.925554 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b740f7d9-92c8-4924-b086-8943a0a617e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:13.947084 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 674/1357] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:13.983648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d42422-9842-4a4b-893b-a3be9cb73012 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:13.983942 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d42422-9842-4a4b-893b-a3be9cb73012 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:13.984217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d42422-9842-4a4b-893b-a3be9cb73012 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:13.985580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 684/1358] 10.222.0.35 () {54 vars in 961 bytes} [Fri May 13 21:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:13.991574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd722b-0216-4db9-9b10-f997783b8c2d None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:13.992057 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd722b-0216-4db9-9b10-f997783b8c2d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:13.992432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd722b-0216-4db9-9b10-f997783b8c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:14.040697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-43bd722b-0216-4db9-9b10-f997783b8c2d None None] MFA Rules not processed for user `316ef43c6c304bd3a27e085bf4da8a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:14.041906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-43bd722b-0216-4db9-9b10-f997783b8c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:14.121846 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 675/1359] 10.222.0.35 () {58 vars in 1058 bytes} [Fri May 13 21:44:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:14.130104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d42422-9842-4a4b-893b-a3be9cb73012 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:14.143987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d42422-9842-4a4b-893b-a3be9cb73012 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:14.152341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8d513-41a4-4067-a071-c3f3bef90ff2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:14.152341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8d513-41a4-4067-a071-c3f3bef90ff2 service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:14.152341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8d513-41a4-4067-a071-c3f3bef90ff2 service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:14.152341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8d513-41a4-4067-a071-c3f3bef90ff2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:14.161302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8d513-41a4-4067-a071-c3f3bef90ff2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:14.163238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8d513-41a4-4067-a071-c3f3bef90ff2 service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:14.173275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 685/1360] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:44:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:44:16.629374 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a458db-4213-4206-96d9-e1412ca1855a None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:16.629861 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a458db-4213-4206-96d9-e1412ca1855a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:16.629963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a458db-4213-4206-96d9-e1412ca1855a None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:16.631621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 676/1361] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:44:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:16.637208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd4c63-6aae-4c59-9bd8-eb08e1742382 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:16.637640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd4c63-6aae-4c59-9bd8-eb08e1742382 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:16.637761 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd4c63-6aae-4c59-9bd8-eb08e1742382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:16.657976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-73bd4c63-6aae-4c59-9bd8-eb08e1742382 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:16.659481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-73bd4c63-6aae-4c59-9bd8-eb08e1742382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:16.667531 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 686/1362] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:16.713462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a458db-4213-4206-96d9-e1412ca1855a None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:16.728018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a458db-4213-4206-96d9-e1412ca1855a None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:16.733720 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c0d22-1b54-4ead-8e6c-8d376e6aa3aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:16.734476 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c0d22-1b54-4ead-8e6c-8d376e6aa3aa service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:16.734903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c0d22-1b54-4ead-8e6c-8d376e6aa3aa service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:16.735284 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c0d22-1b54-4ead-8e6c-8d376e6aa3aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:16.744114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c0d22-1b54-4ead-8e6c-8d376e6aa3aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:16.746058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c0d22-1b54-4ead-8e6c-8d376e6aa3aa service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:16.757054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 677/1363] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:44:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:44:19.276920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddc56e-76e2-4215-8b56-de85992129cc None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:19.277398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddc56e-76e2-4215-8b56-de85992129cc None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:19.277491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddc56e-76e2-4215-8b56-de85992129cc None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:19.279075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 687/1364] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:19.289078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ffe2b-b567-4a4c-a1e3-aa19d688eb9a None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:19.289078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ffe2b-b567-4a4c-a1e3-aa19d688eb9a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:19.289078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ffe2b-b567-4a4c-a1e3-aa19d688eb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:19.310561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-873ffe2b-b567-4a4c-a1e3-aa19d688eb9a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:19.312100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-873ffe2b-b567-4a4c-a1e3-aa19d688eb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:19.320888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 678/1365] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:19.377100 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0ef75-55d5-469c-89f0-cf80f070bd0c None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:19.377388 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0ef75-55d5-469c-89f0-cf80f070bd0c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:19.377665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0ef75-55d5-469c-89f0-cf80f070bd0c None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:19.382084 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 688/1366] 10.222.0.35 () {54 vars in 961 bytes} [Fri May 13 21:44:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:19.391019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c280a3d-e8fa-43a5-be67-963e729a6fa7 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:19.391710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c280a3d-e8fa-43a5-be67-963e729a6fa7 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:19.392202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c280a3d-e8fa-43a5-be67-963e729a6fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:19.411133 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-2c280a3d-e8fa-43a5-be67-963e729a6fa7 None None] MFA Rules not processed for user `316ef43c6c304bd3a27e085bf4da8a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:19.412280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2c280a3d-e8fa-43a5-be67-963e729a6fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:19.421500 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 679/1367] 10.222.0.35 () {58 vars in 1058 bytes} [Fri May 13 21:44:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:44:19.429697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0ef75-55d5-469c-89f0-cf80f070bd0c None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:19.444178 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0ef75-55d5-469c-89f0-cf80f070bd0c None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:19.449438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7325e3-0cb1-477b-8f8d-8db2662ed14f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:19.450062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7325e3-0cb1-477b-8f8d-8db2662ed14f service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:19.450343 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7325e3-0cb1-477b-8f8d-8db2662ed14f service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:19.450617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7325e3-0cb1-477b-8f8d-8db2662ed14f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:19.458936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7325e3-0cb1-477b-8f8d-8db2662ed14f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:19.460550 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7325e3-0cb1-477b-8f8d-8db2662ed14f service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:19.469690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 689/1368] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:44:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:44:21.865455 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13c390-9daa-4e7d-b501-f0eaa98d80ef None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:21.867471 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13c390-9daa-4e7d-b501-f0eaa98d80ef None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:21.867471 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13c390-9daa-4e7d-b501-f0eaa98d80ef None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:21.867746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 680/1369] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:21.873199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f10ee9-eaed-44aa-a831-b66e2914eff2 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:21.873617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f10ee9-eaed-44aa-a831-b66e2914eff2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:21.873757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f10ee9-eaed-44aa-a831-b66e2914eff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:21.891724 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e8f10ee9-eaed-44aa-a831-b66e2914eff2 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:21.893142 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e8f10ee9-eaed-44aa-a831-b66e2914eff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:21.901835 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 690/1370] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:21.947634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13c390-9daa-4e7d-b501-f0eaa98d80ef None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:21.963935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13c390-9daa-4e7d-b501-f0eaa98d80ef None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:21.969817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fef39e-e522-416c-a014-2f98dfaa436b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:21.970631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fef39e-e522-416c-a014-2f98dfaa436b service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:21.970975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fef39e-e522-416c-a014-2f98dfaa436b service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:21.971259 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fef39e-e522-416c-a014-2f98dfaa436b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:21.983273 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fef39e-e522-416c-a014-2f98dfaa436b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:21.984877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fef39e-e522-416c-a014-2f98dfaa436b service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:21.995579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 681/1371] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:44:22.126267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae739d-5e28-475a-a01f-2cd102936e63 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:22.126899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae739d-5e28-475a-a01f-2cd102936e63 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:22.127208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae739d-5e28-475a-a01f-2cd102936e63 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:22.129295 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 691/1372] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:44:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:22.135938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d37e7-8abb-4e4d-b4eb-0a100b62bbce None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:22.136329 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d37e7-8abb-4e4d-b4eb-0a100b62bbce None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:22.136515 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d37e7-8abb-4e4d-b4eb-0a100b62bbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:22.160693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-640d37e7-8abb-4e4d-b4eb-0a100b62bbce None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:22.162300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-640d37e7-8abb-4e4d-b4eb-0a100b62bbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:22.171685 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 682/1373] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:22.481616 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3910b6f-67f1-4ea8-8b14-cfc702a79078 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:22.481616 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3910b6f-67f1-4ea8-8b14-cfc702a79078 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:22.481616 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3910b6f-67f1-4ea8-8b14-cfc702a79078 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:22.482754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 692/1374] 10.222.0.35 () {54 vars in 963 bytes} [Fri May 13 21:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:22.488673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342716b2-bb64-4049-a493-f09a3112fe12 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:22.488673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342716b2-bb64-4049-a493-f09a3112fe12 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:22.489236 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342716b2-bb64-4049-a493-f09a3112fe12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:22.538137 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-342716b2-bb64-4049-a493-f09a3112fe12 None None] MFA Rules not processed for user `86c4a81ce97b4fd78889fae6c36bf6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:22.539380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-342716b2-bb64-4049-a493-f09a3112fe12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:22.599851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 683/1375] 10.222.0.35 () {58 vars in 1060 bytes} [Fri May 13 21:44:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:22.607387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3910b6f-67f1-4ea8-8b14-cfc702a79078 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:22.617820 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3910b6f-67f1-4ea8-8b14-cfc702a79078 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:22.621809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230ae0d-1604-41b1-9ca4-100decbd943e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:22.622441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230ae0d-1604-41b1-9ca4-100decbd943e service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:22.622753 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230ae0d-1604-41b1-9ca4-100decbd943e service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:22.623036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230ae0d-1604-41b1-9ca4-100decbd943e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:22.633979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230ae0d-1604-41b1-9ca4-100decbd943e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:22.635446 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230ae0d-1604-41b1-9ca4-100decbd943e service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:22.645508 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 693/1376] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:44:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:44:24.282683 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cc37c-a941-4473-b1b7-4249fda74bc3 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:24.283285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cc37c-a941-4473-b1b7-4249fda74bc3 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:24.283285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cc37c-a941-4473-b1b7-4249fda74bc3 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:24.284698 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 684/1377] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:24.290326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bf8be-e6dd-4b92-b297-17c2fcd68ce9 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:24.290854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bf8be-e6dd-4b92-b297-17c2fcd68ce9 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:24.290948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bf8be-e6dd-4b92-b297-17c2fcd68ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:24.309433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-856bf8be-e6dd-4b92-b297-17c2fcd68ce9 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:24.310660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-856bf8be-e6dd-4b92-b297-17c2fcd68ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:24.319340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 694/1378] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:44:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:24.363378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cc37c-a941-4473-b1b7-4249fda74bc3 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:24.378501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cc37c-a941-4473-b1b7-4249fda74bc3 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:24.384901 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba5c26-5eb5-4dbd-8424-abab1577b343 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:24.385536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba5c26-5eb5-4dbd-8424-abab1577b343 service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:24.385814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba5c26-5eb5-4dbd-8424-abab1577b343 service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:24.386093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba5c26-5eb5-4dbd-8424-abab1577b343 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:24.394987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba5c26-5eb5-4dbd-8424-abab1577b343 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:24.396600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba5c26-5eb5-4dbd-8424-abab1577b343 service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:24.406033 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 685/1379] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:44:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:44:26.660420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898eb8c-0922-4aa4-8e39-c75ecc0f5a9a None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:26.661347 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898eb8c-0922-4aa4-8e39-c75ecc0f5a9a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:26.661638 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898eb8c-0922-4aa4-8e39-c75ecc0f5a9a None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:26.663114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 695/1380] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:44:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:44:26.669112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb3f5c-3036-4f21-922b-2ca659e91b95 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:26.669572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb3f5c-3036-4f21-922b-2ca659e91b95 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:26.669665 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb3f5c-3036-4f21-922b-2ca659e91b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:26.687870 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-cdeb3f5c-3036-4f21-922b-2ca659e91b95 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:26.689325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-cdeb3f5c-3036-4f21-922b-2ca659e91b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:26.697973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 686/1381] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:26.742221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898eb8c-0922-4aa4-8e39-c75ecc0f5a9a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:26.758178 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898eb8c-0922-4aa4-8e39-c75ecc0f5a9a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:26.764578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142a383-08b4-4143-89d5-94afac531cee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:26.765325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142a383-08b4-4143-89d5-94afac531cee service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:26.765710 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142a383-08b4-4143-89d5-94afac531cee service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:26.766089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142a383-08b4-4143-89d5-94afac531cee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:26.777195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142a383-08b4-4143-89d5-94afac531cee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:26.778935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142a383-08b4-4143-89d5-94afac531cee service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:26.790754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 696/1382] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:44:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:44:29.032701 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376f32-f810-4297-aa94-59dea31453c4 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:29.033352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376f32-f810-4297-aa94-59dea31453c4 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:29.033352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376f32-f810-4297-aa94-59dea31453c4 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:29.035342 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 687/1383] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:44:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:29.041657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39784e-4e97-4f4a-aa46-9877da1eb49f None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:29.042259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39784e-4e97-4f4a-aa46-9877da1eb49f None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:29.042259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39784e-4e97-4f4a-aa46-9877da1eb49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:29.065276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-cd39784e-4e97-4f4a-aa46-9877da1eb49f None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:29.066574 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-cd39784e-4e97-4f4a-aa46-9877da1eb49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:29.074945 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 697/1384] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:44:29.116749 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25376f32-f810-4297-aa94-59dea31453c4 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:29.131573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25376f32-f810-4297-aa94-59dea31453c4 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:29.137713 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35866ac-8852-40fa-a044-0523b3b16547 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:29.138364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35866ac-8852-40fa-a044-0523b3b16547 service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:29.138644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35866ac-8852-40fa-a044-0523b3b16547 service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:29.138950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35866ac-8852-40fa-a044-0523b3b16547 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:29.148489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35866ac-8852-40fa-a044-0523b3b16547 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:29.150200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35866ac-8852-40fa-a044-0523b3b16547 service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:29.162144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 688/1385] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:44:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:44:31.378982 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6ca4c-649c-4e51-8c8f-f1f4a18c2dd3 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:31.379671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6ca4c-649c-4e51-8c8f-f1f4a18c2dd3 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:31.379671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6ca4c-649c-4e51-8c8f-f1f4a18c2dd3 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:31.380916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 698/1386] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:31.386260 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b77298-d990-4e6f-ac59-36727efd7f44 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:31.386750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b77298-d990-4e6f-ac59-36727efd7f44 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:31.386863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b77298-d990-4e6f-ac59-36727efd7f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:31.405197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-41b77298-d990-4e6f-ac59-36727efd7f44 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:31.406351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-41b77298-d990-4e6f-ac59-36727efd7f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:31.414186 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 689/1387] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:44:31.458099 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6ca4c-649c-4e51-8c8f-f1f4a18c2dd3 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:31.472203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6ca4c-649c-4e51-8c8f-f1f4a18c2dd3 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:31.477603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef67b3-fbed-432e-b6d7-31e3c3b43162 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:31.478234 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef67b3-fbed-432e-b6d7-31e3c3b43162 service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:31.478534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef67b3-fbed-432e-b6d7-31e3c3b43162 service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:31.478882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef67b3-fbed-432e-b6d7-31e3c3b43162 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:31.487657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef67b3-fbed-432e-b6d7-31e3c3b43162 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:31.489237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef67b3-fbed-432e-b6d7-31e3c3b43162 service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:31.498870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 699/1388] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:44:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:44:34.348595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeeaf6-02d7-4144-8d2a-bd41c05dbb10 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:34.349140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeeaf6-02d7-4144-8d2a-bd41c05dbb10 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:34.349140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeeaf6-02d7-4144-8d2a-bd41c05dbb10 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:34.350757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 690/1389] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:44:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:34.355988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2ca1-eafc-4dbd-b8f9-ede72de9f1b0 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:34.356415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2ca1-eafc-4dbd-b8f9-ede72de9f1b0 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:34.356657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2ca1-eafc-4dbd-b8f9-ede72de9f1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:34.374943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-532b2ca1-eafc-4dbd-b8f9-ede72de9f1b0 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:34.376200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-532b2ca1-eafc-4dbd-b8f9-ede72de9f1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:34.384338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 700/1390] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:44:34.431062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeeaf6-02d7-4144-8d2a-bd41c05dbb10 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:34.448545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeeaf6-02d7-4144-8d2a-bd41c05dbb10 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:34.455526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21e35f-2506-42ba-992d-7714f01676c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:34.457206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21e35f-2506-42ba-992d-7714f01676c8 service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:34.457637 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21e35f-2506-42ba-992d-7714f01676c8 service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:34.458037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21e35f-2506-42ba-992d-7714f01676c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:34.469235 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21e35f-2506-42ba-992d-7714f01676c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:34.471291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21e35f-2506-42ba-992d-7714f01676c8 service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:34.484015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 691/1391] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:44:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:44:36.841269 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dff9a-3e70-4b00-a6c6-986807aa5186 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:36.841887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dff9a-3e70-4b00-a6c6-986807aa5186 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:36.841887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dff9a-3e70-4b00-a6c6-986807aa5186 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:36.843248 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 701/1392] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:36.849005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c78ee-ba39-4f14-bab1-ae2e470346cc None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:36.849422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c78ee-ba39-4f14-bab1-ae2e470346cc None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:36.849631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c78ee-ba39-4f14-bab1-ae2e470346cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:36.866878 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-dc7c78ee-ba39-4f14-bab1-ae2e470346cc None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:36.868081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-dc7c78ee-ba39-4f14-bab1-ae2e470346cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:36.876179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 692/1393] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:36.919950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dff9a-3e70-4b00-a6c6-986807aa5186 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:36.934032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dff9a-3e70-4b00-a6c6-986807aa5186 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:36.939578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac69ba-3481-447a-9988-5db95b88b68b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:36.940727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac69ba-3481-447a-9988-5db95b88b68b service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:36.941012 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac69ba-3481-447a-9988-5db95b88b68b service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:36.941288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac69ba-3481-447a-9988-5db95b88b68b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:36.949363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac69ba-3481-447a-9988-5db95b88b68b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:36.950976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac69ba-3481-447a-9988-5db95b88b68b service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:36.961253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 702/1394] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:44:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:44:38.211897 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615174b-2062-4a67-a385-0f832e28c155 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:38.212444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615174b-2062-4a67-a385-0f832e28c155 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:38.212444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615174b-2062-4a67-a385-0f832e28c155 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:38.213837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 693/1395] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:38.219226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d34ea-6665-4f78-a344-0789d8648ee3 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:38.219697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d34ea-6665-4f78-a344-0789d8648ee3 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:38.219917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d34ea-6665-4f78-a344-0789d8648ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:38.237965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e30d34ea-6665-4f78-a344-0789d8648ee3 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:38.239186 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e30d34ea-6665-4f78-a344-0789d8648ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:38.247205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 703/1396] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:38.302110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95685c07-8f5e-4de7-818a-cbd19f43c616 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:38.302399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95685c07-8f5e-4de7-818a-cbd19f43c616 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:38.302681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95685c07-8f5e-4de7-818a-cbd19f43c616 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:38.304078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 694/1397] 10.222.0.35 () {54 vars in 962 bytes} [Fri May 13 21:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:38.308950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0044b9-e9c0-4c04-ba4c-e9e8a7f23df9 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:38.309586 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0044b9-e9c0-4c04-ba4c-e9e8a7f23df9 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:38.309866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0044b9-e9c0-4c04-ba4c-e9e8a7f23df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:38.355992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-0a0044b9-e9c0-4c04-ba4c-e9e8a7f23df9 None None] MFA Rules not processed for user `cad57b046f3a43978d1c98bcc1a76d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:38.357164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-0a0044b9-e9c0-4c04-ba4c-e9e8a7f23df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:38.414064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 704/1398] 10.222.0.35 () {58 vars in 1059 bytes} [Fri May 13 21:44:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:44:38.421449 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95685c07-8f5e-4de7-818a-cbd19f43c616 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:38.432030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95685c07-8f5e-4de7-818a-cbd19f43c616 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:38.435555 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150c61c-0b2c-4af2-b499-ab10ee2c636b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad57b046f3a43978d1c98bcc1a76d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:38.436179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150c61c-0b2c-4af2-b499-ab10ee2c636b service glance] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:38.436460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150c61c-0b2c-4af2-b499-ab10ee2c636b service glance] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:38.436738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150c61c-0b2c-4af2-b499-ab10ee2c636b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:38.445962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150c61c-0b2c-4af2-b499-ab10ee2c636b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:38.447425 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150c61c-0b2c-4af2-b499-ab10ee2c636b service glance] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:38.457166 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 695/1399] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:44:38.863596 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5350d8-3732-48a1-80ed-80c8fa2794fa None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:38.863951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5350d8-3732-48a1-80ed-80c8fa2794fa None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:38.864298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5350d8-3732-48a1-80ed-80c8fa2794fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:38.911821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-6c5350d8-3732-48a1-80ed-80c8fa2794fa None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:38.913057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-6c5350d8-3732-48a1-80ed-80c8fa2794fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:38.974101 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 705/1400] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:44:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:44:39.016813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5e623-cd7f-4653-acbf-d24adef59228 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:39.017097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5e623-cd7f-4653-acbf-d24adef59228 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:39.017374 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5e623-cd7f-4653-acbf-d24adef59228 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:39.018881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 696/1401] 10.222.0.35 () {54 vars in 942 bytes} [Fri May 13 21:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:39.025871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a102d0-f9be-420b-a6a1-36dedbc6859d None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:39.026162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a102d0-f9be-420b-a6a1-36dedbc6859d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:39.026442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a102d0-f9be-420b-a6a1-36dedbc6859d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:39.075456 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-94a102d0-f9be-420b-a6a1-36dedbc6859d None None] MFA Rules not processed for user `4f9e07ed0a084c82b946aaf585de16fd`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:39.076558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-94a102d0-f9be-420b-a6a1-36dedbc6859d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:39.152520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 706/1402] 10.222.0.35 () {58 vars in 1039 bytes} [Fri May 13 21:44:39 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:39.158612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5e623-cd7f-4653-acbf-d24adef59228 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:39.172189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5e623-cd7f-4653-acbf-d24adef59228 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:39.177741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66a50b-c83a-486c-96fd-c2579e9a7b45 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:39.178373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66a50b-c83a-486c-96fd-c2579e9a7b45 service swift] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:39.178660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66a50b-c83a-486c-96fd-c2579e9a7b45 service swift] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:39.178970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66a50b-c83a-486c-96fd-c2579e9a7b45 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:39.186939 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66a50b-c83a-486c-96fd-c2579e9a7b45 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:39.190568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66a50b-c83a-486c-96fd-c2579e9a7b45 service swift] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:39.197038 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 697/1403] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:44:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:44:39.202859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d57c2-45c0-41d1-892e-d1639af8d06b None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:39.203198 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d57c2-45c0-41d1-892e-d1639af8d06b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:39.203426 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d57c2-45c0-41d1-892e-d1639af8d06b None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:39.204800 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 707/1404] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:39.210129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3b598-e87e-45ad-bc28-c2f46a80f689 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:39.210437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3b598-e87e-45ad-bc28-c2f46a80f689 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:39.210657 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3b598-e87e-45ad-bc28-c2f46a80f689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:39.229864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-30e3b598-e87e-45ad-bc28-c2f46a80f689 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:39.231541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-30e3b598-e87e-45ad-bc28-c2f46a80f689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:39.240156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 698/1405] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:44:39.289588 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d57c2-45c0-41d1-892e-d1639af8d06b None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:39.306421 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d57c2-45c0-41d1-892e-d1639af8d06b None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:39.312213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8c150-1454-4d08-9640-3e4ffd95a4a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:39.312933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8c150-1454-4d08-9640-3e4ffd95a4a8 service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:39.313287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8c150-1454-4d08-9640-3e4ffd95a4a8 service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:39.313637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8c150-1454-4d08-9640-3e4ffd95a4a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:39.325595 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8c150-1454-4d08-9640-3e4ffd95a4a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:39.327213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8c150-1454-4d08-9640-3e4ffd95a4a8 service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:39.338464 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 708/1406] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:44:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:44:41.601624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cf1a4-1543-433f-ad6f-6ede2912c617 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:41.602169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cf1a4-1543-433f-ad6f-6ede2912c617 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:41.602169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cf1a4-1543-433f-ad6f-6ede2912c617 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:41.603732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 699/1407] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:44:41.608707 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdbe6b-5a46-4193-9219-de30b2ee2a61 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:41.609132 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdbe6b-5a46-4193-9219-de30b2ee2a61 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:41.609268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdbe6b-5a46-4193-9219-de30b2ee2a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:41.628141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-f7cdbe6b-5a46-4193-9219-de30b2ee2a61 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:44:41.629611 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f7cdbe6b-5a46-4193-9219-de30b2ee2a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:44:41.638405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 709/1408] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:44:41.682666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cf1a4-1543-433f-ad6f-6ede2912c617 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:44:41.698275 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cf1a4-1543-433f-ad6f-6ede2912c617 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:44:41.703704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96e03d-5711-4d18-bed7-2d37aa3c9a4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:44:41.704587 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96e03d-5711-4d18-bed7-2d37aa3c9a4f service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:44:41.705072 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96e03d-5711-4d18-bed7-2d37aa3c9a4f service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:44:41.705349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96e03d-5711-4d18-bed7-2d37aa3c9a4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:44:41.716621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96e03d-5711-4d18-bed7-2d37aa3c9a4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:44:41.718207 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96e03d-5711-4d18-bed7-2d37aa3c9a4f service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:44:41.728105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 700/1409] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:44:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:02.744924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea0a51-b817-4e36-9ce6-bb789944769b None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:02.745805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea0a51-b817-4e36-9ce6-bb789944769b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:02.745805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea0a51-b817-4e36-9ce6-bb789944769b None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:02.747151 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 710/1410] 10.222.0.35 () {54 vars in 918 bytes} [Fri May 13 21:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:02.753161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee09300-526a-436c-87ea-86b64b734b84 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:02.753658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee09300-526a-436c-87ea-86b64b734b84 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:02.753763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee09300-526a-436c-87ea-86b64b734b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:02.771159 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-6ee09300-526a-436c-87ea-86b64b734b84 None None] MFA Rules not processed for user `8124ab8abcc640c5a48b3e0216afa48d`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:02.772356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-6ee09300-526a-436c-87ea-86b64b734b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:02.780724 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 701/1411] 10.222.0.35 () {58 vars in 1015 bytes} [Fri May 13 21:45:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:02.794328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c9773-f50f-4f3b-9871-d3cdb39337f6 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:02.794623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c9773-f50f-4f3b-9871-d3cdb39337f6 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:02.794927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c9773-f50f-4f3b-9871-d3cdb39337f6 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:02.796276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 711/1412] 10.222.0.35 () {54 vars in 918 bytes} [Fri May 13 21:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:02.801325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca2be6-54c6-498f-8d7a-9721513c7478 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:02.801614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca2be6-54c6-498f-8d7a-9721513c7478 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:02.801922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca2be6-54c6-498f-8d7a-9721513c7478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:02.819691 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-0dca2be6-54c6-498f-8d7a-9721513c7478 None None] MFA Rules not processed for user `316ef43c6c304bd3a27e085bf4da8a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:02.820870 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0dca2be6-54c6-498f-8d7a-9721513c7478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:02.829879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 702/1413] 10.222.0.35 () {58 vars in 1015 bytes} [Fri May 13 21:45:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:05.544173 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f2658-c475-492c-bdcf-4cf4efe3b08d None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:05.544741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f2658-c475-492c-bdcf-4cf4efe3b08d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:05.544741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f2658-c475-492c-bdcf-4cf4efe3b08d None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:05.546176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:05.551482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bc305-41ac-4a72-995a-af3698c315fd None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:05.551881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bc305-41ac-4a72-995a-af3698c315fd None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:05.552047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bc305-41ac-4a72-995a-af3698c315fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:05.569759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-488bc305-41ac-4a72-995a-af3698c315fd None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:05.570981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-488bc305-41ac-4a72-995a-af3698c315fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:05.578593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:05.620225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f2658-c475-492c-bdcf-4cf4efe3b08d None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:05.635158 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f2658-c475-492c-bdcf-4cf4efe3b08d None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:05.640579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc4324-cb70-4bba-9f7f-0d82ae099558 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:05.641211 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc4324-cb70-4bba-9f7f-0d82ae099558 service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:05.641496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc4324-cb70-4bba-9f7f-0d82ae099558 service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:05.641780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc4324-cb70-4bba-9f7f-0d82ae099558 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:05.650478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc4324-cb70-4bba-9f7f-0d82ae099558 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:05.652113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc4324-cb70-4bba-9f7f-0d82ae099558 service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:05.661840 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 713/1416] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:06.639334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9fdd3-4ea4-4866-ac7f-c042667a39bc None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:06.639908 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9fdd3-4ea4-4866-ac7f-c042667a39bc None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:06.639908 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9fdd3-4ea4-4866-ac7f-c042667a39bc None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:06.641295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 21:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:06.646821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d5577-5f23-4d7b-9a44-1f31860ab83c None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:06.647284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d5577-5f23-4d7b-9a44-1f31860ab83c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:06.647395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d5577-5f23-4d7b-9a44-1f31860ab83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:06.665683 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-690d5577-5f23-4d7b-9a44-1f31860ab83c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:06.666916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-690d5577-5f23-4d7b-9a44-1f31860ab83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:06.674547 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 21:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:45:06.831319 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9fdd3-4ea4-4866-ac7f-c042667a39bc None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:06.845692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9fdd3-4ea4-4866-ac7f-c042667a39bc None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:06.851605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4690262-75cb-4641-ba3f-c768e080d77a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:06.852234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4690262-75cb-4641-ba3f-c768e080d77a service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:06.852522 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4690262-75cb-4641-ba3f-c768e080d77a service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:06.852800 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4690262-75cb-4641-ba3f-c768e080d77a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:06.860692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4690262-75cb-4641-ba3f-c768e080d77a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:06.862283 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4690262-75cb-4641-ba3f-c768e080d77a service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:06.871945 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 705/1419] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:07.815083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62454bb-c1f5-43a9-8226-0ae29d9dd048 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:07.815663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62454bb-c1f5-43a9-8226-0ae29d9dd048 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:07.815663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62454bb-c1f5-43a9-8226-0ae29d9dd048 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:07.817047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 21:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:07.821940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d9efc-bc76-43b5-b2b7-c962e4c979c4 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:07.822373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d9efc-bc76-43b5-b2b7-c962e4c979c4 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:07.822497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d9efc-bc76-43b5-b2b7-c962e4c979c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:07.839870 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-201d9efc-bc76-43b5-b2b7-c962e4c979c4 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:07.841047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-201d9efc-bc76-43b5-b2b7-c962e4c979c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:07.849129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 21:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:07.997036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62454bb-c1f5-43a9-8226-0ae29d9dd048 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:08.011034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62454bb-c1f5-43a9-8226-0ae29d9dd048 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:08.016280 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbc3eb-1cbf-445e-a921-5239ca353811 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:08.016905 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbc3eb-1cbf-445e-a921-5239ca353811 service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:08.017184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbc3eb-1cbf-445e-a921-5239ca353811 service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:08.017456 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbc3eb-1cbf-445e-a921-5239ca353811 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:08.025868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbc3eb-1cbf-445e-a921-5239ca353811 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:08.027475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbc3eb-1cbf-445e-a921-5239ca353811 service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:08.037086 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 716/1422] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:08.948489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aba36a-633e-429b-b5a0-d6f2bd83d56a None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:08.949041 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aba36a-633e-429b-b5a0-d6f2bd83d56a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:08.949041 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aba36a-633e-429b-b5a0-d6f2bd83d56a None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:08.950408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 21:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:08.955343 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489bd9a-02ee-4980-b212-76503e7e4410 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:08.955749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489bd9a-02ee-4980-b212-76503e7e4410 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:08.955935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489bd9a-02ee-4980-b212-76503e7e4410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:08.973194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7489bd9a-02ee-4980-b212-76503e7e4410 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:08.974363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7489bd9a-02ee-4980-b212-76503e7e4410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:08.981878 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 21:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:09.136862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aba36a-633e-429b-b5a0-d6f2bd83d56a None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:09.151048 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aba36a-633e-429b-b5a0-d6f2bd83d56a None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:09.156510 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048e827-1562-473e-839c-b172fff08fe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:09.157141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e827-1562-473e-839c-b172fff08fe6 service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:09.157427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e827-1562-473e-839c-b172fff08fe6 service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:09.157709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e827-1562-473e-839c-b172fff08fe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:09.166311 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048e827-1562-473e-839c-b172fff08fe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:09.167949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048e827-1562-473e-839c-b172fff08fe6 service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:09.177660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 708/1425] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:10.100483 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd245a-ced9-4f2d-ae03-477defcca31a None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:10.101007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd245a-ced9-4f2d-ae03-477defcca31a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:10.101007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd245a-ced9-4f2d-ae03-477defcca31a None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:10.102404 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 21:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:10.107254 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6631e7-a2c4-4c58-98c6-93b3b3bf2b27 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:10.107664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6631e7-a2c4-4c58-98c6-93b3b3bf2b27 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:10.107814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6631e7-a2c4-4c58-98c6-93b3b3bf2b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:10.126097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-4d6631e7-a2c4-4c58-98c6-93b3b3bf2b27 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:10.127333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4d6631e7-a2c4-4c58-98c6-93b3b3bf2b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:10.134986 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 21:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:10.287602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd245a-ced9-4f2d-ae03-477defcca31a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:10.301358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd245a-ced9-4f2d-ae03-477defcca31a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:10.306548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49575a4a-32aa-4298-903e-8a67ac11943d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:10.307192 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49575a4a-32aa-4298-903e-8a67ac11943d service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:10.307474 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49575a4a-32aa-4298-903e-8a67ac11943d service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:10.307745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49575a4a-32aa-4298-903e-8a67ac11943d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:10.315543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49575a4a-32aa-4298-903e-8a67ac11943d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:10.317116 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49575a4a-32aa-4298-903e-8a67ac11943d service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:10.326098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 719/1428] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:11.183326 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5295407-e299-40f3-8b0e-ce39340a9d19 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:11.183772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5295407-e299-40f3-8b0e-ce39340a9d19 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:11.183889 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5295407-e299-40f3-8b0e-ce39340a9d19 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:11.185245 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 21:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:11.190207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86815d55-ca91-447e-9025-e1fbe7967bf1 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:11.190641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86815d55-ca91-447e-9025-e1fbe7967bf1 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:11.190813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86815d55-ca91-447e-9025-e1fbe7967bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:11.211191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-86815d55-ca91-447e-9025-e1fbe7967bf1 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:11.213454 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-86815d55-ca91-447e-9025-e1fbe7967bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:11.221820 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 21:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:11.369763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5295407-e299-40f3-8b0e-ce39340a9d19 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:11.384438 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5295407-e299-40f3-8b0e-ce39340a9d19 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:11.389691 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3f26c-e006-41e7-9de9-ad23d356566a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:11.390317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f26c-e006-41e7-9de9-ad23d356566a service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:11.390597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f26c-e006-41e7-9de9-ad23d356566a service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:11.390905 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f26c-e006-41e7-9de9-ad23d356566a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:11.399278 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3f26c-e006-41e7-9de9-ad23d356566a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:11.400851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3f26c-e006-41e7-9de9-ad23d356566a service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:11.410058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 711/1431] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:12.280372 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c352fc8-9839-47bf-aaa0-75b073be949a None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:12.280904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c352fc8-9839-47bf-aaa0-75b073be949a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:12.280904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c352fc8-9839-47bf-aaa0-75b073be949a None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:12.282237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 21:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:12.287738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56274f6f-55ff-4293-a2af-99bf227f0221 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:12.288133 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56274f6f-55ff-4293-a2af-99bf227f0221 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:12.288286 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56274f6f-55ff-4293-a2af-99bf227f0221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:12.306227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-56274f6f-55ff-4293-a2af-99bf227f0221 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:12.307443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-56274f6f-55ff-4293-a2af-99bf227f0221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:12.315071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 21:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:12.463233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c352fc8-9839-47bf-aaa0-75b073be949a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:12.478014 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c352fc8-9839-47bf-aaa0-75b073be949a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:12.483281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4647e7-51bf-4b92-8128-f670c4f90c79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:12.483904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4647e7-51bf-4b92-8128-f670c4f90c79 service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:12.484184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4647e7-51bf-4b92-8128-f670c4f90c79 service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:12.484459 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4647e7-51bf-4b92-8128-f670c4f90c79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:12.492388 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4647e7-51bf-4b92-8128-f670c4f90c79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:12.494007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4647e7-51bf-4b92-8128-f670c4f90c79 service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:12.503600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 722/1434] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:13.372229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294d3e4-cb77-4d0a-bc7d-7c11b59d5413 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:13.372811 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294d3e4-cb77-4d0a-bc7d-7c11b59d5413 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:13.372811 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294d3e4-cb77-4d0a-bc7d-7c11b59d5413 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:13.374148 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 21:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:13.379576 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4022d57-933e-42ae-bad4-38e943c8c04c None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:13.380011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4022d57-933e-42ae-bad4-38e943c8c04c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:13.380115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4022d57-933e-42ae-bad4-38e943c8c04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:13.397564 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a4022d57-933e-42ae-bad4-38e943c8c04c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:13.398791 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a4022d57-933e-42ae-bad4-38e943c8c04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:13.406650 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 21:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:13.553998 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294d3e4-cb77-4d0a-bc7d-7c11b59d5413 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:13.568413 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294d3e4-cb77-4d0a-bc7d-7c11b59d5413 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:13.573772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfba2fa-df60-4307-9e65-a6abbac2d7f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:13.574410 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfba2fa-df60-4307-9e65-a6abbac2d7f1 service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:13.574694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfba2fa-df60-4307-9e65-a6abbac2d7f1 service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:13.575006 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfba2fa-df60-4307-9e65-a6abbac2d7f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:13.583116 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfba2fa-df60-4307-9e65-a6abbac2d7f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:13.584700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfba2fa-df60-4307-9e65-a6abbac2d7f1 service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:13.593878 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 714/1437] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:14.465321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe252a7-7e36-4143-8bca-e25d08a3a4df None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:14.465865 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe252a7-7e36-4143-8bca-e25d08a3a4df None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:14.465865 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe252a7-7e36-4143-8bca-e25d08a3a4df None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:14.467431 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 21:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:14.472518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb291fce-17e7-4302-8432-38ddbf67b4cd None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:14.472953 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb291fce-17e7-4302-8432-38ddbf67b4cd None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:14.473062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb291fce-17e7-4302-8432-38ddbf67b4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:14.491191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-cb291fce-17e7-4302-8432-38ddbf67b4cd None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:14.492388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-cb291fce-17e7-4302-8432-38ddbf67b4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:14.500399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 21:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:45:14.650277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe252a7-7e36-4143-8bca-e25d08a3a4df None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:14.664814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe252a7-7e36-4143-8bca-e25d08a3a4df None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:14.670116 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ced794-40f5-4000-88c6-dd217a1e3faf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:14.670772 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ced794-40f5-4000-88c6-dd217a1e3faf service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:14.671061 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ced794-40f5-4000-88c6-dd217a1e3faf service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:14.671340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ced794-40f5-4000-88c6-dd217a1e3faf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:14.679863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ced794-40f5-4000-88c6-dd217a1e3faf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:14.681441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ced794-40f5-4000-88c6-dd217a1e3faf service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:14.690930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 725/1440] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:16.773131 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd93b0-eddb-4851-be44-b3a261ecaefd None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:16.773695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd93b0-eddb-4851-be44-b3a261ecaefd None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:16.773695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd93b0-eddb-4851-be44-b3a261ecaefd None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:16.775119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:16.780328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6a52d-e5e7-4b29-b931-51ff2fbecbe8 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:16.780781 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6a52d-e5e7-4b29-b931-51ff2fbecbe8 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:16.780874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6a52d-e5e7-4b29-b931-51ff2fbecbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:16.798821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-f7b6a52d-e5e7-4b29-b931-51ff2fbecbe8 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:16.800092 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f7b6a52d-e5e7-4b29-b931-51ff2fbecbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:16.807959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:16.846079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd93b0-eddb-4851-be44-b3a261ecaefd None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:16.856501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd93b0-eddb-4851-be44-b3a261ecaefd None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:16.859952 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7817dc-4cb3-481b-80c3-66a98faa2723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:16.860580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7817dc-4cb3-481b-80c3-66a98faa2723 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:16.860861 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7817dc-4cb3-481b-80c3-66a98faa2723 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:16.861148 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7817dc-4cb3-481b-80c3-66a98faa2723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:16.869758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7817dc-4cb3-481b-80c3-66a98faa2723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:16.871428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7817dc-4cb3-481b-80c3-66a98faa2723 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:16.881002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 717/1443] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:19.254556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e91905-89e1-4cdd-888c-b48a39323482 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:19.255153 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e91905-89e1-4cdd-888c-b48a39323482 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:19.255153 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e91905-89e1-4cdd-888c-b48a39323482 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:19.256580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:19.261731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55add1ec-0ccd-4d41-8699-2930c2b87649 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:19.262163 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55add1ec-0ccd-4d41-8699-2930c2b87649 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:19.262320 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55add1ec-0ccd-4d41-8699-2930c2b87649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:19.280053 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-55add1ec-0ccd-4d41-8699-2930c2b87649 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:19.281272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-55add1ec-0ccd-4d41-8699-2930c2b87649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:19.289601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:45:19.325993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e91905-89e1-4cdd-888c-b48a39323482 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:19.337056 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e91905-89e1-4cdd-888c-b48a39323482 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:19.340567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fb35e-0e8f-4fc4-8a30-135b0e33f4f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:19.341183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fb35e-0e8f-4fc4-8a30-135b0e33f4f2 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:19.341462 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fb35e-0e8f-4fc4-8a30-135b0e33f4f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:19.341731 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fb35e-0e8f-4fc4-8a30-135b0e33f4f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:19.351924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fb35e-0e8f-4fc4-8a30-135b0e33f4f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:19.353813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fb35e-0e8f-4fc4-8a30-135b0e33f4f2 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:19.363223 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 728/1446] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:21.998444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a19a78-9763-4816-8540-63c813280fd0 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:21.998996 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a19a78-9763-4816-8540-63c813280fd0 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:21.998996 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a19a78-9763-4816-8540-63c813280fd0 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:22.000384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:22.006258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da08ffb-d04e-4622-b09f-b771747f49aa None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:22.006685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da08ffb-d04e-4622-b09f-b771747f49aa None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:22.006859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da08ffb-d04e-4622-b09f-b771747f49aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:22.024536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-8da08ffb-d04e-4622-b09f-b771747f49aa None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:22.025725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-8da08ffb-d04e-4622-b09f-b771747f49aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:22.033587 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:45:22.071981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a19a78-9763-4816-8540-63c813280fd0 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:22.083023 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a19a78-9763-4816-8540-63c813280fd0 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:22.086526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6b958-8fa7-4505-be37-03bafebd596f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:22.087191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6b958-8fa7-4505-be37-03bafebd596f service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:22.087474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6b958-8fa7-4505-be37-03bafebd596f service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:22.087748 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6b958-8fa7-4505-be37-03bafebd596f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:22.096378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6b958-8fa7-4505-be37-03bafebd596f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:22.097805 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6b958-8fa7-4505-be37-03bafebd596f service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:22.106901 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 720/1449] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:24.426175 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a10b7-e759-4d0f-b39a-008db3900ba9 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:24.426773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a10b7-e759-4d0f-b39a-008db3900ba9 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:24.427130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a10b7-e759-4d0f-b39a-008db3900ba9 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:24.429711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:45:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:24.435279 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636afdc-339a-4fa5-9426-6f4544b59fd4 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:24.435706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636afdc-339a-4fa5-9426-6f4544b59fd4 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:24.435863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636afdc-339a-4fa5-9426-6f4544b59fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:24.453859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-6636afdc-339a-4fa5-9426-6f4544b59fd4 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:24.455117 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-6636afdc-339a-4fa5-9426-6f4544b59fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:24.463450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:24.501301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a10b7-e759-4d0f-b39a-008db3900ba9 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:24.511893 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a10b7-e759-4d0f-b39a-008db3900ba9 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:24.515457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325aa92-8591-4cce-b51a-32d0a96d889b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:24.516083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325aa92-8591-4cce-b51a-32d0a96d889b service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:24.516365 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325aa92-8591-4cce-b51a-32d0a96d889b service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:24.516639 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325aa92-8591-4cce-b51a-32d0a96d889b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:24.525787 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325aa92-8591-4cce-b51a-32d0a96d889b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:24.527267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325aa92-8591-4cce-b51a-32d0a96d889b service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:24.536954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 731/1452] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:26.786907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e5e24-91a4-4eb1-bb27-b7c452bc5293 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:26.787482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e5e24-91a4-4eb1-bb27-b7c452bc5293 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:26.787482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e5e24-91a4-4eb1-bb27-b7c452bc5293 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:26.788839 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:26.794126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12727b4d-9e9b-4d82-b624-35abc385e71c None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:26.794595 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12727b4d-9e9b-4d82-b624-35abc385e71c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:26.794799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12727b4d-9e9b-4d82-b624-35abc385e71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:26.812127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-12727b4d-9e9b-4d82-b624-35abc385e71c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:26.813321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-12727b4d-9e9b-4d82-b624-35abc385e71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:26.821552 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:26.858042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e5e24-91a4-4eb1-bb27-b7c452bc5293 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:26.869196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e5e24-91a4-4eb1-bb27-b7c452bc5293 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:26.872932 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35614f0f-295d-4c01-b2d5-2577538679a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:26.873549 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35614f0f-295d-4c01-b2d5-2577538679a0 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:26.873827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35614f0f-295d-4c01-b2d5-2577538679a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:26.874097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35614f0f-295d-4c01-b2d5-2577538679a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:26.883210 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35614f0f-295d-4c01-b2d5-2577538679a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:26.884621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35614f0f-295d-4c01-b2d5-2577538679a0 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:26.893616 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 723/1455] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:29.192991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b39b83-9b4b-460c-8669-f1e7f6542522 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:29.193498 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b39b83-9b4b-460c-8669-f1e7f6542522 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:29.193606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b39b83-9b4b-460c-8669-f1e7f6542522 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:29.194941 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:29.200390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cd44f-fb15-440b-9f5f-aae2575fd9a7 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:29.200845 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cd44f-fb15-440b-9f5f-aae2575fd9a7 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:29.200952 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cd44f-fb15-440b-9f5f-aae2575fd9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:29.217962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-d29cd44f-fb15-440b-9f5f-aae2575fd9a7 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:29.219181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d29cd44f-fb15-440b-9f5f-aae2575fd9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:29.226896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:29.264678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b39b83-9b4b-460c-8669-f1e7f6542522 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:29.275237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b39b83-9b4b-460c-8669-f1e7f6542522 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:29.278967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb31c2-db81-48da-b158-6fc947a93fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:29.279600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb31c2-db81-48da-b158-6fc947a93fca service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:29.279886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb31c2-db81-48da-b158-6fc947a93fca service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:29.280164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb31c2-db81-48da-b158-6fc947a93fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:29.289019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb31c2-db81-48da-b158-6fc947a93fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:29.290449 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb31c2-db81-48da-b158-6fc947a93fca service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:29.300165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 734/1458] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:31.561078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d909f-93f7-42b8-b72b-4a3f6c29524d None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:31.561666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d909f-93f7-42b8-b72b-4a3f6c29524d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:31.561666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d909f-93f7-42b8-b72b-4a3f6c29524d None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:31.563066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:31.568959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274183d4-c79f-4ee0-a61e-7e9e09d1ec1f None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:31.569387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274183d4-c79f-4ee0-a61e-7e9e09d1ec1f None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:31.569519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274183d4-c79f-4ee0-a61e-7e9e09d1ec1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:31.587532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-274183d4-c79f-4ee0-a61e-7e9e09d1ec1f None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:31.588708 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-274183d4-c79f-4ee0-a61e-7e9e09d1ec1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:31.596781 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:45:31.633475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d909f-93f7-42b8-b72b-4a3f6c29524d None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:31.643848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d909f-93f7-42b8-b72b-4a3f6c29524d None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:31.647236 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc230c1-4980-423e-9571-8940b04b33bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:31.647854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc230c1-4980-423e-9571-8940b04b33bc service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:31.648134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc230c1-4980-423e-9571-8940b04b33bc service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:31.648407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc230c1-4980-423e-9571-8940b04b33bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:31.656922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc230c1-4980-423e-9571-8940b04b33bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:31.658865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc230c1-4980-423e-9571-8940b04b33bc service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:31.667936 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 726/1461] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:33.920308 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af1fd3-b8ff-4f81-a6ed-e6914762ac9f None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:33.920895 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af1fd3-b8ff-4f81-a6ed-e6914762ac9f None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:33.920895 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af1fd3-b8ff-4f81-a6ed-e6914762ac9f None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:33.922258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:33.927618 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262c091-db88-4714-be2d-0c3d7f24141d None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:33.928007 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262c091-db88-4714-be2d-0c3d7f24141d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:33.928182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262c091-db88-4714-be2d-0c3d7f24141d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:33.946405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-8262c091-db88-4714-be2d-0c3d7f24141d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:33.947492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8262c091-db88-4714-be2d-0c3d7f24141d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:33.956803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:33.993384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af1fd3-b8ff-4f81-a6ed-e6914762ac9f None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:34.004776 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af1fd3-b8ff-4f81-a6ed-e6914762ac9f None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:34.008310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30176b1a-64ad-4f84-8e84-e3d860401034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:34.008935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30176b1a-64ad-4f84-8e84-e3d860401034 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:34.009220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30176b1a-64ad-4f84-8e84-e3d860401034 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:34.009499 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30176b1a-64ad-4f84-8e84-e3d860401034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:34.018805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30176b1a-64ad-4f84-8e84-e3d860401034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:34.020230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30176b1a-64ad-4f84-8e84-e3d860401034 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:34.029664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 737/1464] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:36.269342 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93674ce1-2a8c-4ee0-b71d-ca78567b0e49 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:36.269848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93674ce1-2a8c-4ee0-b71d-ca78567b0e49 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:36.269848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93674ce1-2a8c-4ee0-b71d-ca78567b0e49 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:36.271222 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:45:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:36.276455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5078d97-8c2e-406d-a017-bb0aa4aaf617 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:36.276869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5078d97-8c2e-406d-a017-bb0aa4aaf617 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:36.277012 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5078d97-8c2e-406d-a017-bb0aa4aaf617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:36.294014 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-f5078d97-8c2e-406d-a017-bb0aa4aaf617 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:36.295238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f5078d97-8c2e-406d-a017-bb0aa4aaf617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:36.302924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:45:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:36.339513 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93674ce1-2a8c-4ee0-b71d-ca78567b0e49 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:36.349922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93674ce1-2a8c-4ee0-b71d-ca78567b0e49 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:36.353445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818659c-794c-40eb-b683-620486d63cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:36.354079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818659c-794c-40eb-b683-620486d63cbc service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:36.354364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818659c-794c-40eb-b683-620486d63cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:36.354641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818659c-794c-40eb-b683-620486d63cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:36.363367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818659c-794c-40eb-b683-620486d63cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:36.364825 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818659c-794c-40eb-b683-620486d63cbc service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:36.374374 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 729/1467] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:45:38.581915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20d49b-6417-412d-b179-e3a7cba630da None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:38.582487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20d49b-6417-412d-b179-e3a7cba630da None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:38.582487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20d49b-6417-412d-b179-e3a7cba630da None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:38.584300 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:45:38.590026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf368b-0ac3-4aee-9b19-7fe0d898da0e None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:38.590502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf368b-0ac3-4aee-9b19-7fe0d898da0e None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:38.590630 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf368b-0ac3-4aee-9b19-7fe0d898da0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:38.608185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-f5cf368b-0ac3-4aee-9b19-7fe0d898da0e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:45:38.609384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f5cf368b-0ac3-4aee-9b19-7fe0d898da0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:45:38.617647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:45:38.652439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20d49b-6417-412d-b179-e3a7cba630da None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:45:38.663422 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20d49b-6417-412d-b179-e3a7cba630da None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:45:38.667640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf8524-560e-4783-899a-f9c1878b8db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:45:38.668275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf8524-560e-4783-899a-f9c1878b8db5 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:45:38.668564 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf8524-560e-4783-899a-f9c1878b8db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:45:38.668842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf8524-560e-4783-899a-f9c1878b8db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:45:38.678367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf8524-560e-4783-899a-f9c1878b8db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:45:38.679850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf8524-560e-4783-899a-f9c1878b8db5 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:45:38.689383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 740/1470] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:45:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:46:11.620746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712215fc-3f2b-423f-a523-ffccda0666f1 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:11.621347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712215fc-3f2b-423f-a523-ffccda0666f1 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:11.621347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712215fc-3f2b-423f-a523-ffccda0666f1 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:11.622657 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 731/1471] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:46:11.628209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5741b-21b1-433b-b10c-d3a2d0e2ce5a None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:11.628689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5741b-21b1-433b-b10c-d3a2d0e2ce5a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:11.628796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5741b-21b1-433b-b10c-d3a2d0e2ce5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:11.646577 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-8eb5741b-21b1-433b-b10c-d3a2d0e2ce5a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:46:11.647794 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-8eb5741b-21b1-433b-b10c-d3a2d0e2ce5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:46:11.655534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 741/1472] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:46:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:46:11.701349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3d90d-d3fa-419a-b508-63bd26730462 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:11.701646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3d90d-d3fa-419a-b508-63bd26730462 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:11.701912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3d90d-d3fa-419a-b508-63bd26730462 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:11.703301 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 732/1473] 10.222.0.35 () {54 vars in 962 bytes} [Fri May 13 21:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:46:11.708663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6d5de-4ee6-4410-8edb-1725ea7598ee None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:11.708946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6d5de-4ee6-4410-8edb-1725ea7598ee None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:11.709217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6d5de-4ee6-4410-8edb-1725ea7598ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:11.726937 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-5ff6d5de-4ee6-4410-8edb-1725ea7598ee None None] MFA Rules not processed for user `cad57b046f3a43978d1c98bcc1a76d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:46:11.728140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-5ff6d5de-4ee6-4410-8edb-1725ea7598ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:46:11.734492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 742/1474] 10.222.0.35 () {58 vars in 1059 bytes} [Fri May 13 21:46:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:46:11.741941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3d90d-d3fa-419a-b508-63bd26730462 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:46:11.752923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3d90d-d3fa-419a-b508-63bd26730462 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:46:11.756337 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2676c6-9cbb-47f7-b056-9313fb45f703 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad57b046f3a43978d1c98bcc1a76d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:46:11.756945 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2676c6-9cbb-47f7-b056-9313fb45f703 service glance] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:11.757227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2676c6-9cbb-47f7-b056-9313fb45f703 service glance] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:11.757501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2676c6-9cbb-47f7-b056-9313fb45f703 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:11.765977 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2676c6-9cbb-47f7-b056-9313fb45f703 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:46:11.767425 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2676c6-9cbb-47f7-b056-9313fb45f703 service glance] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:46:11.776423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 733/1475] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:46:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:46:14.062824 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca3a67-9c97-4baf-89c5-e53746cb7fb7 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:14.063419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca3a67-9c97-4baf-89c5-e53746cb7fb7 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:14.063419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca3a67-9c97-4baf-89c5-e53746cb7fb7 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:14.065381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 743/1476] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:46:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:46:14.070574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306d21a-0b29-4ce8-9979-9d01d5aa2936 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:14.071023 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306d21a-0b29-4ce8-9979-9d01d5aa2936 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:14.071171 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306d21a-0b29-4ce8-9979-9d01d5aa2936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:14.088205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-4306d21a-0b29-4ce8-9979-9d01d5aa2936 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:46:14.089379 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4306d21a-0b29-4ce8-9979-9d01d5aa2936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:46:14.096920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 734/1477] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:46:14.119826 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca3a67-9c97-4baf-89c5-e53746cb7fb7 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:46:14.132640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca3a67-9c97-4baf-89c5-e53746cb7fb7 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:46:14.137007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97897638-23c8-4320-b2de-70417632cf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:46:14.137632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97897638-23c8-4320-b2de-70417632cf0e admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:14.137909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97897638-23c8-4320-b2de-70417632cf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:14.138179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97897638-23c8-4320-b2de-70417632cf0e admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:14.138934 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97897638-23c8-4320-b2de-70417632cf0e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:46:14.140662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: WARNING py.warnings [None req-97897638-23c8-4320-b2de-70417632cf0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 21:46:14.140662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: warnings.warn(msg) May 13 21:46:14.140662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: May 13 21:46:14.141842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97897638-23c8-4320-b2de-70417632cf0e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:46:14.160420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 744/1478] 10.222.0.35 () {56 vars in 1108 bytes} [Fri May 13 21:46:14 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:46:16.377761 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdf0e8-5b5a-45c4-83a7-090aa8c24419 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:16.378277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdf0e8-5b5a-45c4-83a7-090aa8c24419 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:16.378277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdf0e8-5b5a-45c4-83a7-090aa8c24419 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:16.379734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 735/1479] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:46:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:46:16.384930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb8181-3e1a-4011-a78d-17d921b15b59 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:16.385340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb8181-3e1a-4011-a78d-17d921b15b59 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:16.385494 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb8181-3e1a-4011-a78d-17d921b15b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:16.402960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-68cb8181-3e1a-4011-a78d-17d921b15b59 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:46:16.404152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-68cb8181-3e1a-4011-a78d-17d921b15b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:46:16.411935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 745/1480] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:46:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:46:16.460404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdf0e8-5b5a-45c4-83a7-090aa8c24419 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:46:16.474820 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdf0e8-5b5a-45c4-83a7-090aa8c24419 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:46:16.480659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df153292-587a-4c87-848d-c4888d567cb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:46:16.481347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df153292-587a-4c87-848d-c4888d567cb7 service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:16.481781 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df153292-587a-4c87-848d-c4888d567cb7 service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:16.482193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df153292-587a-4c87-848d-c4888d567cb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:16.491803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df153292-587a-4c87-848d-c4888d567cb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:46:16.493543 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df153292-587a-4c87-848d-c4888d567cb7 service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:46:16.503945 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 736/1481] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:46:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:46:18.871354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb6f47-7e14-46c5-aa3a-5e695135d8e4 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:18.871882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb6f47-7e14-46c5-aa3a-5e695135d8e4 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:18.871882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb6f47-7e14-46c5-aa3a-5e695135d8e4 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:18.873275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 746/1482] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:46:18.879062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93885626-536d-4310-a322-b61240072d6d None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:18.879488 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93885626-536d-4310-a322-b61240072d6d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:18.879626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93885626-536d-4310-a322-b61240072d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:18.897483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-93885626-536d-4310-a322-b61240072d6d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:46:18.898683 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-93885626-536d-4310-a322-b61240072d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:46:18.906468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 737/1483] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:46:18.941633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb6f47-7e14-46c5-aa3a-5e695135d8e4 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:46:18.951940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb6f47-7e14-46c5-aa3a-5e695135d8e4 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:46:18.955954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e3ada-c64b-4fe0-8d82-20d680627cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:46:18.956575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e3ada-c64b-4fe0-8d82-20d680627cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:18.956857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e3ada-c64b-4fe0-8d82-20d680627cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:18.957135 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e3ada-c64b-4fe0-8d82-20d680627cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:18.965840 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e3ada-c64b-4fe0-8d82-20d680627cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:46:18.967321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e3ada-c64b-4fe0-8d82-20d680627cc2 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:46:18.976493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 747/1484] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:46:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:46:21.274048 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616e815-ee36-4bc3-b9e0-89ccf43061fc None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:21.274518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616e815-ee36-4bc3-b9e0-89ccf43061fc None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:21.274631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616e815-ee36-4bc3-b9e0-89ccf43061fc None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:21.276013 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 738/1485] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:46:21.281950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc183e-e7fa-4f28-b7cd-12d59a5f63d6 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:21.282415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc183e-e7fa-4f28-b7cd-12d59a5f63d6 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:21.282522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc183e-e7fa-4f28-b7cd-12d59a5f63d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:21.300074 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-acbc183e-e7fa-4f28-b7cd-12d59a5f63d6 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:46:21.301265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-acbc183e-e7fa-4f28-b7cd-12d59a5f63d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:46:21.308951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 748/1486] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:46:21.351171 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616e815-ee36-4bc3-b9e0-89ccf43061fc None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:46:21.363892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616e815-ee36-4bc3-b9e0-89ccf43061fc None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:46:21.367336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2f0e5-8d90-4155-9692-4c916da9a08b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:46:21.367955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2f0e5-8d90-4155-9692-4c916da9a08b service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:21.368234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2f0e5-8d90-4155-9692-4c916da9a08b service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:21.368505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2f0e5-8d90-4155-9692-4c916da9a08b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:21.378366 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2f0e5-8d90-4155-9692-4c916da9a08b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:46:21.379869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2f0e5-8d90-4155-9692-4c916da9a08b service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:46:21.389194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 739/1487] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:46:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:46:23.835270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f92ed0-f181-45e2-a441-a35defe3c350 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:23.835838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f92ed0-f181-45e2-a441-a35defe3c350 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:23.835838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f92ed0-f181-45e2-a441-a35defe3c350 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:23.837256 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 749/1488] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:46:23.842529 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff38c7-f2e7-48be-8b97-a99a45b0de1b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:23.842966 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff38c7-f2e7-48be-8b97-a99a45b0de1b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:23.843123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff38c7-f2e7-48be-8b97-a99a45b0de1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:23.860794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-27ff38c7-f2e7-48be-8b97-a99a45b0de1b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:46:23.861988 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-27ff38c7-f2e7-48be-8b97-a99a45b0de1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:46:23.869627 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 740/1489] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:46:23.909552 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f92ed0-f181-45e2-a441-a35defe3c350 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:46:23.920235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f92ed0-f181-45e2-a441-a35defe3c350 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:46:23.924189 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9e361-7540-4216-ae19-7c652f9c9bde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:46:23.924811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9e361-7540-4216-ae19-7c652f9c9bde service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:23.925091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9e361-7540-4216-ae19-7c652f9c9bde service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:23.925365 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9e361-7540-4216-ae19-7c652f9c9bde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:23.933814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9e361-7540-4216-ae19-7c652f9c9bde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:46:23.935389 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9e361-7540-4216-ae19-7c652f9c9bde service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:46:23.944913 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 750/1490] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:46:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:46:26.304402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe317ea1-0825-43b9-9c89-ed37b87b6c83 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:26.305003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe317ea1-0825-43b9-9c89-ed37b87b6c83 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:26.305003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe317ea1-0825-43b9-9c89-ed37b87b6c83 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:26.306284 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 741/1491] 10.222.0.35 () {54 vars in 926 bytes} [Fri May 13 21:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:46:26.311623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d16a9-daec-46c5-bc0f-786b92a74ff6 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:26.312058 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d16a9-daec-46c5-bc0f-786b92a74ff6 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:26.312189 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d16a9-daec-46c5-bc0f-786b92a74ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:26.330321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e17d16a9-daec-46c5-bc0f-786b92a74ff6 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:46:26.331566 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e17d16a9-daec-46c5-bc0f-786b92a74ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:46:26.339414 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 751/1492] 10.222.0.35 () {58 vars in 1023 bytes} [Fri May 13 21:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:46:26.378473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe317ea1-0825-43b9-9c89-ed37b87b6c83 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:46:26.388796 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe317ea1-0825-43b9-9c89-ed37b87b6c83 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:46:26.392292 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fe49b-2ae7-4d4a-b25b-d0b00268b3d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:46:26.392940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fe49b-2ae7-4d4a-b25b-d0b00268b3d1 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:46:26.393222 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fe49b-2ae7-4d4a-b25b-d0b00268b3d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:46:26.393517 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fe49b-2ae7-4d4a-b25b-d0b00268b3d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:46:26.402402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fe49b-2ae7-4d4a-b25b-d0b00268b3d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:46:26.403858 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fe49b-2ae7-4d4a-b25b-d0b00268b3d1 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:46:26.413447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 742/1493] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:46:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:47:53.426784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39e9d-d01b-403a-9789-fe9e988c56b2 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:53.427352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39e9d-d01b-403a-9789-fe9e988c56b2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:53.427352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39e9d-d01b-403a-9789-fe9e988c56b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:53.445584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-fef39e9d-d01b-403a-9789-fe9e988c56b2 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:47:53.446805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-fef39e9d-d01b-403a-9789-fe9e988c56b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:47:53.455267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 752/1494] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:47:53.467062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a1234-7c8d-4395-9c99-4e60251e8b52 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:53.467659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a1234-7c8d-4395-9c99-4e60251e8b52 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:53.467659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a1234-7c8d-4395-9c99-4e60251e8b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:53.485714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-c47a1234-7c8d-4395-9c99-4e60251e8b52 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:47:53.486963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c47a1234-7c8d-4395-9c99-4e60251e8b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:47:53.495084 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 743/1495] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:47:53.500249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef39e9d-d01b-403a-9789-fe9e988c56b2 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:47:53.510024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef39e9d-d01b-403a-9789-fe9e988c56b2 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:47:53.514539 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80573cf7-22a5-4ed3-9381-e41bf103ffed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:47:53.515710 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80573cf7-22a5-4ed3-9381-e41bf103ffed admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:53.515990 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80573cf7-22a5-4ed3-9381-e41bf103ffed admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:53.516259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80573cf7-22a5-4ed3-9381-e41bf103ffed admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:53.516970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80573cf7-22a5-4ed3-9381-e41bf103ffed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:47:53.517270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80573cf7-22a5-4ed3-9381-e41bf103ffed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:47:53.518275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: WARNING py.warnings [None req-80573cf7-22a5-4ed3-9381-e41bf103ffed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 13 21:47:53.518275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: warnings.warn(msg) May 13 21:47:53.518275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: May 13 21:47:53.519181 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80573cf7-22a5-4ed3-9381-e41bf103ffed admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:47:53.530277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 753/1496] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:47:53.538218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a1234-7c8d-4395-9c99-4e60251e8b52 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:47:53.547203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a1234-7c8d-4395-9c99-4e60251e8b52 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:47:53.551750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9653e41e-0c6c-4096-90ad-c661fbbe4d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:47:53.552391 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653e41e-0c6c-4096-90ad-c661fbbe4d08 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:53.552676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653e41e-0c6c-4096-90ad-c661fbbe4d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:53.552949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653e41e-0c6c-4096-90ad-c661fbbe4d08 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:53.553624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9653e41e-0c6c-4096-90ad-c661fbbe4d08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:47:53.555402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: WARNING py.warnings [None req-9653e41e-0c6c-4096-90ad-c661fbbe4d08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 21:47:53.555402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: warnings.warn(msg) May 13 21:47:53.555402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: May 13 21:47:53.556611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9653e41e-0c6c-4096-90ad-c661fbbe4d08 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:47:53.594639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 744/1497] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:47:53 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:47:53.595950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80573cf7-22a5-4ed3-9381-e41bf103ffed admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:47:53.605699 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80573cf7-22a5-4ed3-9381-e41bf103ffed admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:47:53.658945 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af15201-42f9-443c-957f-d7721ff53c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:47:53.659597 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af15201-42f9-443c-957f-d7721ff53c67 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:53.659877 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af15201-42f9-443c-957f-d7721ff53c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:53.660151 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af15201-42f9-443c-957f-d7721ff53c67 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:53.660833 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af15201-42f9-443c-957f-d7721ff53c67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:47:53.661871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: WARNING py.warnings [None req-1af15201-42f9-443c-957f-d7721ff53c67 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 21:47:53.661871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: warnings.warn(msg) May 13 21:47:53.661871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: May 13 21:47:53.663103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af15201-42f9-443c-957f-d7721ff53c67 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:47:53.681452 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 754/1498] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:47:53 2022] POST /identity/v3/users => generated 289 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:47:53.685594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9653e41e-0c6c-4096-90ad-c661fbbe4d08 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:47:53.696254 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9653e41e-0c6c-4096-90ad-c661fbbe4d08 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:47:53.700682 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13d418-48ef-4aa1-9534-8bb5ffdead1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:47:53.701325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13d418-48ef-4aa1-9534-8bb5ffdead1a admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:53.701605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13d418-48ef-4aa1-9534-8bb5ffdead1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:53.701877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13d418-48ef-4aa1-9534-8bb5ffdead1a admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:53.703252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13d418-48ef-4aa1-9534-8bb5ffdead1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:47:53.704264 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: WARNING py.warnings [None req-6b13d418-48ef-4aa1-9534-8bb5ffdead1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 13 21:47:53.704264 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: warnings.warn(msg) May 13 21:47:53.704264 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: May 13 21:47:53.705137 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13d418-48ef-4aa1-9534-8bb5ffdead1a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:47:53.717438 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 745/1499] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:47:53.721254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af15201-42f9-443c-957f-d7721ff53c67 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:47:53.730679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af15201-42f9-443c-957f-d7721ff53c67 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:47:53.735696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec68176e-db8b-4195-8e63-40f61a52eca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:47:53.736334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68176e-db8b-4195-8e63-40f61a52eca0 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:53.736614 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68176e-db8b-4195-8e63-40f61a52eca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:53.736889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68176e-db8b-4195-8e63-40f61a52eca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:53.737594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec68176e-db8b-4195-8e63-40f61a52eca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:47:53.738627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: WARNING py.warnings [None req-ec68176e-db8b-4195-8e63-40f61a52eca0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 13 21:47:53.738627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: warnings.warn(msg) May 13 21:47:53.738627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: May 13 21:47:53.739535 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec68176e-db8b-4195-8e63-40f61a52eca0 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:47:53.753227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 755/1500] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:47:53.757206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13d418-48ef-4aa1-9534-8bb5ffdead1a admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:47:53.766832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13d418-48ef-4aa1-9534-8bb5ffdead1a admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:47:53.771788 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee56a2b-57f0-4bcb-aeef-9efa42f33f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:47:53.772649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee56a2b-57f0-4bcb-aeef-9efa42f33f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:53.772925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee56a2b-57f0-4bcb-aeef-9efa42f33f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:53.773196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee56a2b-57f0-4bcb-aeef-9efa42f33f7b admin admin] PATH_INFO: `/v3/projects/816f85fb40304248a4c7dcea1c579931/users/a6b89ccbd2814d89b6b30b8fb561802e/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:53.787305 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee56a2b-57f0-4bcb-aeef-9efa42f33f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=816f85fb40304248a4c7dcea1c579931, user_id=a6b89ccbd2814d89b6b30b8fb561802e, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:47:53.788328 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: WARNING py.warnings [None req-1ee56a2b-57f0-4bcb-aeef-9efa42f33f7b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 21:47:53.788328 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: warnings.warn(msg) May 13 21:47:53.788328 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: May 13 21:47:53.790288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee56a2b-57f0-4bcb-aeef-9efa42f33f7b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:47:53.803131 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 746/1501] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:47:53 2022] PUT /identity/v3/projects/816f85fb40304248a4c7dcea1c579931/users/a6b89ccbd2814d89b6b30b8fb561802e/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:47:53.815586 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bc996-5937-4319-871b-4b5b4b2f3068 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:53.815890 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bc996-5937-4319-871b-4b5b4b2f3068 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:53.816165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bc996-5937-4319-871b-4b5b4b2f3068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:53.846229 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-120bc996-5937-4319-871b-4b5b4b2f3068 None None] MFA Rules not processed for user `a6b89ccbd2814d89b6b30b8fb561802e`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:47:53.847452 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-120bc996-5937-4319-871b-4b5b4b2f3068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:47:53.907857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 756/1502] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:47:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:47:53.921389 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa36bb-d838-4c7d-8b77-4e27c3d9b427 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:53.921678 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa36bb-d838-4c7d-8b77-4e27c3d9b427 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:53.921987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa36bb-d838-4c7d-8b77-4e27c3d9b427 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:53.923378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 747/1503] 10.222.0.35 () {54 vars in 963 bytes} [Fri May 13 21:47:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:47:53.928255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66957be-63dc-4f23-93f9-856560341164 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:53.928546 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66957be-63dc-4f23-93f9-856560341164 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:53.928821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66957be-63dc-4f23-93f9-856560341164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:53.946458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e66957be-63dc-4f23-93f9-856560341164 None None] MFA Rules not processed for user `86c4a81ce97b4fd78889fae6c36bf6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:47:53.947664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e66957be-63dc-4f23-93f9-856560341164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:47:53.995971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 757/1504] 10.222.0.35 () {58 vars in 1060 bytes} [Fri May 13 21:47:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:47:54.004042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa36bb-d838-4c7d-8b77-4e27c3d9b427 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:47:54.014848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa36bb-d838-4c7d-8b77-4e27c3d9b427 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:47:54.017849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3603a3b-b087-4f15-bac1-67d975351422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:47:54.018479 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3603a3b-b087-4f15-bac1-67d975351422 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:54.018787 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3603a3b-b087-4f15-bac1-67d975351422 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:54.019063 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3603a3b-b087-4f15-bac1-67d975351422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:54.027133 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3603a3b-b087-4f15-bac1-67d975351422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:47:54.028545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3603a3b-b087-4f15-bac1-67d975351422 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:47:54.037017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 748/1505] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:47:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:47:54.078869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e9a66-01bc-4edf-9e81-80e51f0873c1 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:54.079160 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e9a66-01bc-4edf-9e81-80e51f0873c1 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:54.079438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e9a66-01bc-4edf-9e81-80e51f0873c1 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:54.080719 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 758/1506] 10.222.0.35 () {52 vars in 829 bytes} [Fri May 13 21:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:47:54.084294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3603a3b-b087-4f15-bac1-67d975351422 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:47:54.152811 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3603a3b-b087-4f15-bac1-67d975351422 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:47:54.157911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:47:54.158801 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:54.159082 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:54.159352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] PATH_INFO: `/v3/users/a6b89ccbd2814d89b6b30b8fb561802e` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:54.160521 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6b89ccbd2814d89b6b30b8fb561802e)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:47:54.161533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: WARNING py.warnings [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 21:47:54.161533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: warnings.warn(msg) May 13 21:47:54.161533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: May 13 21:47:54.162788 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:47:54.225137 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6b89ccbd2814d89b6b30b8fb561802e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:47:54.243099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6b89ccbd2814d89b6b30b8fb561802e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:47:54.243767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6b89ccbd2814d89b6b30b8fb561802e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:47:54.244489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6b89ccbd2814d89b6b30b8fb561802e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:47:54.256494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6b89ccbd2814d89b6b30b8fb561802e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:47:54.271382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 749/1507] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:47:54 2022] DELETE /identity/v3/users/a6b89ccbd2814d89b6b30b8fb561802e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:47:54.275730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ee988-789f-4c03-8135-d3a7107b898c None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:54.276018 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ee988-789f-4c03-8135-d3a7107b898c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:54.276293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ee988-789f-4c03-8135-d3a7107b898c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:54.293491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d56ee988-789f-4c03-8135-d3a7107b898c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:47:54.294670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d56ee988-789f-4c03-8135-d3a7107b898c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:47:54.350218 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 759/1508] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:47:54.360148 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:47:54.361568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:47:54.415960 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be47587-b1cd-43bf-9fa8-86fb325d282a admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:47:54.419555 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccba1b-af3c-4ce3-9911-08c59fac83cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:47:54.420187 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccba1b-af3c-4ce3-9911-08c59fac83cd service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:54.420474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccba1b-af3c-4ce3-9911-08c59fac83cd service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:54.421346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccba1b-af3c-4ce3-9911-08c59fac83cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:54.429974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccba1b-af3c-4ce3-9911-08c59fac83cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:47:54.431444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccba1b-af3c-4ce3-9911-08c59fac83cd service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:47:54.440568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 750/1509] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:47:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:47:54.849423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ee988-789f-4c03-8135-d3a7107b898c None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:47:54.850861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d56ee988-789f-4c03-8135-d3a7107b898c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:47:54.862141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ee988-789f-4c03-8135-d3a7107b898c None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:47:54.867678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b680b-1431-402f-9d18-4928592c9368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:47:54.868605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b680b-1431-402f-9d18-4928592c9368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:47:54.868885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b680b-1431-402f-9d18-4928592c9368 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:47:54.869159 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b680b-1431-402f-9d18-4928592c9368 admin admin] PATH_INFO: `/v3/projects/816f85fb40304248a4c7dcea1c579931` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:47:54.870608 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b680b-1431-402f-9d18-4928592c9368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=816f85fb40304248a4c7dcea1c579931)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:47:54.871885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: WARNING py.warnings [None req-380b680b-1431-402f-9d18-4928592c9368 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 21:47:54.871885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: warnings.warn(msg) May 13 21:47:54.871885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: May 13 21:47:54.873092 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b680b-1431-402f-9d18-4928592c9368 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:47:54.930043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-380b680b-1431-402f-9d18-4928592c9368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '816f85fb40304248a4c7dcea1c579931', 'request_id': 'req-380b680b-1431-402f-9d18-4928592c9368'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:47:54.943729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-380b680b-1431-402f-9d18-4928592c9368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '816f85fb40304248a4c7dcea1c579931', 'request_id': 'req-380b680b-1431-402f-9d18-4928592c9368'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:47:54.954026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-380b680b-1431-402f-9d18-4928592c9368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '816f85fb40304248a4c7dcea1c579931', 'request_id': 'req-380b680b-1431-402f-9d18-4928592c9368'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:47:54.964446 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-380b680b-1431-402f-9d18-4928592c9368 admin admin] The token cache is being invalidate because project 816f85fb40304248a4c7dcea1c579931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:47:54.964734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-380b680b-1431-402f-9d18-4928592c9368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:47:54.965585 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-380b680b-1431-402f-9d18-4928592c9368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:47:54.967268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 760/1510] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:47:54 2022] DELETE /identity/v3/projects/816f85fb40304248a4c7dcea1c579931 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:49:37.347756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc95a5e-68a2-494f-b319-01d89f98550b None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:37.348360 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc95a5e-68a2-494f-b319-01d89f98550b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:37.348360 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc95a5e-68a2-494f-b319-01d89f98550b None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:37.349695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 21:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:49:37.354525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a9394-768b-4f4c-bce6-b62fbcd349d2 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:37.354982 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a9394-768b-4f4c-bce6-b62fbcd349d2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:37.355138 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a9394-768b-4f4c-bce6-b62fbcd349d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:37.373750 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-769a9394-768b-4f4c-bce6-b62fbcd349d2 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:49:37.374989 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-769a9394-768b-4f4c-bce6-b62fbcd349d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:49:37.432634 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 21:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:49:37.598015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc95a5e-68a2-494f-b319-01d89f98550b None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:49:37.599721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8dc95a5e-68a2-494f-b319-01d89f98550b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:49:37.672814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc95a5e-68a2-494f-b319-01d89f98550b None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:49:37.678106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44290c0d-304d-4e46-904f-60c167923263 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:49:37.678761 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44290c0d-304d-4e46-904f-60c167923263 service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:37.679047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44290c0d-304d-4e46-904f-60c167923263 service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:37.679323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44290c0d-304d-4e46-904f-60c167923263 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:37.687057 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44290c0d-304d-4e46-904f-60c167923263 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:49:37.688614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44290c0d-304d-4e46-904f-60c167923263 service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:49:37.697542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 752/1513] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:49:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:49:38.420649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3e481-2a22-45df-b55c-1d8a617064ec None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:38.421191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3e481-2a22-45df-b55c-1d8a617064ec None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:38.421191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3e481-2a22-45df-b55c-1d8a617064ec None None] PATH_INFO: `/v3` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:38.423115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 762/1514] 10.222.0.58 () {54 vars in 925 bytes} [Fri May 13 21:49:38 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:49:38.429753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca385a9f-8eb1-43d0-a38d-230e853ef130 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:38.430191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca385a9f-8eb1-43d0-a38d-230e853ef130 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:38.430299 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca385a9f-8eb1-43d0-a38d-230e853ef130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:38.448212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ca385a9f-8eb1-43d0-a38d-230e853ef130 None None] MFA Rules not processed for user `8124ab8abcc640c5a48b3e0216afa48d`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:49:38.449420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ca385a9f-8eb1-43d0-a38d-230e853ef130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:49:38.510281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 753/1515] 10.222.0.58 () {58 vars in 1015 bytes} [Fri May 13 21:49:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:49:38.593759 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739569c-a6a8-43f2-b6f5-a0c6bb0fbd52 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:38.594547 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739569c-a6a8-43f2-b6f5-a0c6bb0fbd52 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:38.594847 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739569c-a6a8-43f2-b6f5-a0c6bb0fbd52 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:38.596228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 763/1516] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 21:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:49:38.601064 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02727968-85a6-4b08-9ae7-280f988cc75b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:38.601357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02727968-85a6-4b08-9ae7-280f988cc75b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:38.601639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02727968-85a6-4b08-9ae7-280f988cc75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:38.620600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-02727968-85a6-4b08-9ae7-280f988cc75b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:49:38.621794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-02727968-85a6-4b08-9ae7-280f988cc75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:49:38.629592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 754/1517] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 21:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:49:38.783011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739569c-a6a8-43f2-b6f5-a0c6bb0fbd52 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:49:38.796829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739569c-a6a8-43f2-b6f5-a0c6bb0fbd52 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:49:38.802480 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c106f9d-260b-4579-9246-3f350db28c87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:49:38.803139 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106f9d-260b-4579-9246-3f350db28c87 service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:38.803418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106f9d-260b-4579-9246-3f350db28c87 service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:38.803688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106f9d-260b-4579-9246-3f350db28c87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:38.811709 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c106f9d-260b-4579-9246-3f350db28c87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:49:38.813843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c106f9d-260b-4579-9246-3f350db28c87 service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:49:38.823580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 764/1518] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:49:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:49:47.435041 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90993d05-3242-4428-9a39-d15a8bc19707 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:47.435599 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90993d05-3242-4428-9a39-d15a8bc19707 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:47.435599 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90993d05-3242-4428-9a39-d15a8bc19707 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:47.436973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 755/1519] 10.222.0.35 () {54 vars in 960 bytes} [Fri May 13 21:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:49:47.442384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62926aeb-4660-4011-95f0-f00d0322f841 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:47.442801 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62926aeb-4660-4011-95f0-f00d0322f841 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:47.442970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62926aeb-4660-4011-95f0-f00d0322f841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:47.460923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-62926aeb-4660-4011-95f0-f00d0322f841 None None] MFA Rules not processed for user `8124ab8abcc640c5a48b3e0216afa48d`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:49:47.462116 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-62926aeb-4660-4011-95f0-f00d0322f841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:49:47.470475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 765/1520] 10.222.0.35 () {58 vars in 1057 bytes} [Fri May 13 21:49:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:49:47.477461 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90993d05-3242-4428-9a39-d15a8bc19707 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:49:47.490588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90993d05-3242-4428-9a39-d15a8bc19707 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:49:47.495698 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9b052-4275-4235-b546-e283e1aa2056 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8124ab8abcc640c5a48b3e0216afa48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:49:47.496309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9b052-4275-4235-b546-e283e1aa2056 service placement] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:47.496592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9b052-4275-4235-b546-e283e1aa2056 service placement] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:47.496868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9b052-4275-4235-b546-e283e1aa2056 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:47.505280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9b052-4275-4235-b546-e283e1aa2056 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:49:47.506892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9b052-4275-4235-b546-e283e1aa2056 service placement] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:49:47.516143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 756/1521] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:49:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:49:50.115884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6cd59-5a77-46f5-8acd-3e0bd4e4d16d None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:50.116367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6cd59-5a77-46f5-8acd-3e0bd4e4d16d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:50.116475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6cd59-5a77-46f5-8acd-3e0bd4e4d16d None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:50.117823 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 21:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:49:50.122805 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbbea2-14ae-421c-9036-84f4e9a6f6a2 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:50.123193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbbea2-14ae-421c-9036-84f4e9a6f6a2 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:50.123369 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbbea2-14ae-421c-9036-84f4e9a6f6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:50.140704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-47cbbea2-14ae-421c-9036-84f4e9a6f6a2 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:49:50.142432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-47cbbea2-14ae-421c-9036-84f4e9a6f6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:49:50.150154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 21:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:49:50.301226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6cd59-5a77-46f5-8acd-3e0bd4e4d16d None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:49:50.317776 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6cd59-5a77-46f5-8acd-3e0bd4e4d16d None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:49:50.324230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3bcd5-98c2-4e5a-a3e1-c069974d32ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:49:50.324798 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3bcd5-98c2-4e5a-a3e1-c069974d32ef service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:50.325076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3bcd5-98c2-4e5a-a3e1-c069974d32ef service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:50.325451 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3bcd5-98c2-4e5a-a3e1-c069974d32ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:50.334042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3bcd5-98c2-4e5a-a3e1-c069974d32ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:49:50.335642 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3bcd5-98c2-4e5a-a3e1-c069974d32ef service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:49:50.345331 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 767/1524] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:49:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:49:51.229567 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689819f-1658-42a9-8101-2711d5b0ef33 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:51.230085 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689819f-1658-42a9-8101-2711d5b0ef33 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:51.230182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689819f-1658-42a9-8101-2711d5b0ef33 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:51.231534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 21:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:49:51.236746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88900d-249e-4ddd-b4b4-9ab1a8c9d820 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:51.237131 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88900d-249e-4ddd-b4b4-9ab1a8c9d820 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:51.237303 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88900d-249e-4ddd-b4b4-9ab1a8c9d820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:51.256750 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-0b88900d-249e-4ddd-b4b4-9ab1a8c9d820 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:49:51.257015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-0b88900d-249e-4ddd-b4b4-9ab1a8c9d820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:49:51.265417 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 21:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:49:51.420119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689819f-1658-42a9-8101-2711d5b0ef33 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:49:51.434634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689819f-1658-42a9-8101-2711d5b0ef33 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:49:51.440744 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8b39f-ef35-4119-8116-595b3f7221a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:49:51.441378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8b39f-ef35-4119-8116-595b3f7221a6 service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:51.441664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8b39f-ef35-4119-8116-595b3f7221a6 service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:51.441943 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8b39f-ef35-4119-8116-595b3f7221a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:51.449832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8b39f-ef35-4119-8116-595b3f7221a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:49:51.451446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8b39f-ef35-4119-8116-595b3f7221a6 service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:49:51.461151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 759/1527] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:49:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:49:55.466762 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba0873-e7a5-4269-8421-6f2b9527f9e8 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:55.467344 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba0873-e7a5-4269-8421-6f2b9527f9e8 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:55.467344 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba0873-e7a5-4269-8421-6f2b9527f9e8 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:55.468691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:49:55.475053 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a04e6-2e22-4334-9e50-4caf23b600fc None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:55.475594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a04e6-2e22-4334-9e50-4caf23b600fc None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:55.475594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a04e6-2e22-4334-9e50-4caf23b600fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:55.495492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-1a6a04e6-2e22-4334-9e50-4caf23b600fc None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:49:55.496700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-1a6a04e6-2e22-4334-9e50-4caf23b600fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:49:55.504501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:49:55.540929 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba0873-e7a5-4269-8421-6f2b9527f9e8 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:49:55.542605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e6ba0873-e7a5-4269-8421-6f2b9527f9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:49:55.598718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba0873-e7a5-4269-8421-6f2b9527f9e8 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:49:55.602532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f12d9-edbd-4356-8302-84352ae2aee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:49:55.603221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f12d9-edbd-4356-8302-84352ae2aee9 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:55.603550 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f12d9-edbd-4356-8302-84352ae2aee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:55.603784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f12d9-edbd-4356-8302-84352ae2aee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:55.619389 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f12d9-edbd-4356-8302-84352ae2aee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:49:55.620738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f12d9-edbd-4356-8302-84352ae2aee9 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:49:55.632256 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 770/1530] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:49:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:49:57.840685 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509ef6b-a9bd-42cc-89af-300082b359c8 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:57.841256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509ef6b-a9bd-42cc-89af-300082b359c8 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:57.841256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509ef6b-a9bd-42cc-89af-300082b359c8 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:57.842605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:49:57.848437 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d98683-0bd6-4e6a-a765-7a9934ce843e None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:57.848839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d98683-0bd6-4e6a-a765-7a9934ce843e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:57.848991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d98683-0bd6-4e6a-a765-7a9934ce843e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:57.866492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-39d98683-0bd6-4e6a-a765-7a9934ce843e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:49:57.867735 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-39d98683-0bd6-4e6a-a765-7a9934ce843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:49:57.876490 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:49:57.912956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509ef6b-a9bd-42cc-89af-300082b359c8 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:49:57.914647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f509ef6b-a9bd-42cc-89af-300082b359c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:49:57.927149 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509ef6b-a9bd-42cc-89af-300082b359c8 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:49:57.930590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83058ad1-00e1-44a5-b4b7-32843acbbe95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:49:57.931240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83058ad1-00e1-44a5-b4b7-32843acbbe95 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:49:57.931519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83058ad1-00e1-44a5-b4b7-32843acbbe95 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:49:57.931789 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83058ad1-00e1-44a5-b4b7-32843acbbe95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:49:57.940803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83058ad1-00e1-44a5-b4b7-32843acbbe95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:49:57.942221 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83058ad1-00e1-44a5-b4b7-32843acbbe95 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:49:57.951228 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 762/1533] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:50:01.628050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d7793-6df1-4afc-94c6-d489bf527c50 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:50:01.628665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d7793-6df1-4afc-94c6-d489bf527c50 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:50:01.628665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d7793-6df1-4afc-94c6-d489bf527c50 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:50:01.630004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:50:01.635339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea55215-01a3-4d34-8224-2405b296ce3b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:50:01.635804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea55215-01a3-4d34-8224-2405b296ce3b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:50:01.635915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea55215-01a3-4d34-8224-2405b296ce3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:50:01.653843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-0ea55215-01a3-4d34-8224-2405b296ce3b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:50:01.655087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0ea55215-01a3-4d34-8224-2405b296ce3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:50:01.662893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:50:01.932589 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d7793-6df1-4afc-94c6-d489bf527c50 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:50:01.934218 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c89d7793-6df1-4afc-94c6-d489bf527c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:50:01.990097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d7793-6df1-4afc-94c6-d489bf527c50 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:50:01.993844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09d865-0bfc-4042-8636-ab1e5e4d61e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:50:01.994478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09d865-0bfc-4042-8636-ab1e5e4d61e0 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:50:01.994809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09d865-0bfc-4042-8636-ab1e5e4d61e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:50:01.995090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09d865-0bfc-4042-8636-ab1e5e4d61e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:50:02.004510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09d865-0bfc-4042-8636-ab1e5e4d61e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:50:02.005927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09d865-0bfc-4042-8636-ab1e5e4d61e0 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:50:02.015042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 773/1536] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:50:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:50:04.277215 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc514a1-da9c-494a-8a2b-acee0019856b None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:50:04.277771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc514a1-da9c-494a-8a2b-acee0019856b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:50:04.277771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc514a1-da9c-494a-8a2b-acee0019856b None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:50:04.279228 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 21:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:50:04.284513 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8285a-5c20-443b-b1b9-d1258a341a9a None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:50:04.284909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8285a-5c20-443b-b1b9-d1258a341a9a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:50:04.285068 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8285a-5c20-443b-b1b9-d1258a341a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:50:04.303256 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-37e8285a-5c20-443b-b1b9-d1258a341a9a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:50:04.304432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-37e8285a-5c20-443b-b1b9-d1258a341a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:50:04.312084 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 21:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:50:04.581614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc514a1-da9c-494a-8a2b-acee0019856b None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:50:04.583363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8dc514a1-da9c-494a-8a2b-acee0019856b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:50:04.596495 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc514a1-da9c-494a-8a2b-acee0019856b None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:50:04.600014 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd99ee-a563-4b42-8080-f83e9e9f540f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:50:04.600642 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd99ee-a563-4b42-8080-f83e9e9f540f service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:50:04.600926 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd99ee-a563-4b42-8080-f83e9e9f540f service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:50:04.601203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd99ee-a563-4b42-8080-f83e9e9f540f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:50:04.610877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd99ee-a563-4b42-8080-f83e9e9f540f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:50:04.612306 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd99ee-a563-4b42-8080-f83e9e9f540f service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:50:04.621730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 765/1539] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:50:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:33.784943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d2083-75cf-487f-849e-b9ed6139e2ac None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:33.787188 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d2083-75cf-487f-849e-b9ed6139e2ac None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:33.787188 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d2083-75cf-487f-849e-b9ed6139e2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:33.816272 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e09d2083-75cf-487f-849e-b9ed6139e2ac None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:33.818429 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e09d2083-75cf-487f-849e-b9ed6139e2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:33.831876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 775/1540] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:33.845122 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43dd10-b11b-441e-8fe4-98dbfa1f76f0 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:33.845730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43dd10-b11b-441e-8fe4-98dbfa1f76f0 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:33.845730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43dd10-b11b-441e-8fe4-98dbfa1f76f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:33.859965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa98cc7-17ca-4062-a25f-d42ebddef3fe None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:33.860600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa98cc7-17ca-4062-a25f-d42ebddef3fe None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:33.861079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa98cc7-17ca-4062-a25f-d42ebddef3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:33.877322 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-7f43dd10-b11b-441e-8fe4-98dbfa1f76f0 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:33.878790 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-7f43dd10-b11b-441e-8fe4-98dbfa1f76f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:33.896817 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-aaa98cc7-17ca-4062-a25f-d42ebddef3fe None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:33.898496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-aaa98cc7-17ca-4062-a25f-d42ebddef3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:33.898633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 766/1541] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:33.903452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf509-b99d-44e7-9d21-644ceea812e6 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:33.903743 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf509-b99d-44e7-9d21-644ceea812e6 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:33.904033 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf509-b99d-44e7-9d21-644ceea812e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:33.911094 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 776/1542] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:51:33.915086 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa98cc7-17ca-4062-a25f-d42ebddef3fe None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:33.939146 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-c60cf509-b99d-44e7-9d21-644ceea812e6 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:33.940556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c60cf509-b99d-44e7-9d21-644ceea812e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:33.941290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa98cc7-17ca-4062-a25f-d42ebddef3fe None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:33.962766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac97a55-0629-41f8-b4f2-ef719afddb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:33.962766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac97a55-0629-41f8-b4f2-ef719afddb2d admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:33.962766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac97a55-0629-41f8-b4f2-ef719afddb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:33.962766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac97a55-0629-41f8-b4f2-ef719afddb2d admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:33.964317 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac97a55-0629-41f8-b4f2-ef719afddb2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:51:33.964317 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac97a55-0629-41f8-b4f2-ef719afddb2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:33.964712 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac97a55-0629-41f8-b4f2-ef719afddb2d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:33.976027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 767/1543] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:51:33.977022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d008f-9f42-43a0-8ff1-ae97491f8e6b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:33.977552 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d008f-9f42-43a0-8ff1-ae97491f8e6b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:33.977830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d008f-9f42-43a0-8ff1-ae97491f8e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:33.993372 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 777/1544] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:51:33.995305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b4f87-467d-4a2e-99ba-3903bc72d654 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:33.995837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b4f87-467d-4a2e-99ba-3903bc72d654 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:33.996336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b4f87-467d-4a2e-99ba-3903bc72d654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.030107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-3e3d008f-9f42-43a0-8ff1-ae97491f8e6b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:34.030107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-3e3d008f-9f42-43a0-8ff1-ae97491f8e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:34.031433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-8f6b4f87-467d-4a2e-99ba-3903bc72d654 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:34.033609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-8f6b4f87-467d-4a2e-99ba-3903bc72d654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:34.045181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 768/1545] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:34.046468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3bd6d-d8a4-40bb-bfec-75f00f6cd275 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.046811 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3bd6d-d8a4-40bb-bfec-75f00f6cd275 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.047099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3bd6d-d8a4-40bb-bfec-75f00f6cd275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.051670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 778/1546] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:34.052634 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b4f87-467d-4a2e-99ba-3903bc72d654 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.065171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b4f87-467d-4a2e-99ba-3903bc72d654 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.070981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f99ad-acaf-4e8d-b5dd-65da0703e908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.071657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f99ad-acaf-4e8d-b5dd-65da0703e908 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.071944 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f99ad-acaf-4e8d-b5dd-65da0703e908 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.072219 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f99ad-acaf-4e8d-b5dd-65da0703e908 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.072919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f99ad-acaf-4e8d-b5dd-65da0703e908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.074115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: WARNING py.warnings [None req-d98f99ad-acaf-4e8d-b5dd-65da0703e908 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 21:51:34.074115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: warnings.warn(msg) May 13 21:51:34.074115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: May 13 21:51:34.075946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f99ad-acaf-4e8d-b5dd-65da0703e908 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.078660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-c2a3bd6d-d8a4-40bb-bfec-75f00f6cd275 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:34.079996 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c2a3bd6d-d8a4-40bb-bfec-75f00f6cd275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:34.088731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 769/1547] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:34.090117 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8f8f8-d9bb-46f9-8247-5423e2520075 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.090491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8f8f8-d9bb-46f9-8247-5423e2520075 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.090847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8f8f8-d9bb-46f9-8247-5423e2520075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.104904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 779/1548] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:34.106038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a6416-7282-428c-91ba-5b4ac59e8e11 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.106326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a6416-7282-428c-91ba-5b4ac59e8e11 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.106635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a6416-7282-428c-91ba-5b4ac59e8e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.114089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ffb8f8f8-d9bb-46f9-8247-5423e2520075 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:34.115324 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ffb8f8f8-d9bb-46f9-8247-5423e2520075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:34.125240 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-4d2a6416-7282-428c-91ba-5b4ac59e8e11 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:34.126431 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-4d2a6416-7282-428c-91ba-5b4ac59e8e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:34.173463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 770/1549] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:34.174310 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8f8f8-d9bb-46f9-8247-5423e2520075 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.183730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8f8f8-d9bb-46f9-8247-5423e2520075 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.192058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d439980-c802-4b37-a633-8be2ec822da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.192709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d439980-c802-4b37-a633-8be2ec822da9 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.192990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d439980-c802-4b37-a633-8be2ec822da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.193264 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d439980-c802-4b37-a633-8be2ec822da9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.193986 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d439980-c802-4b37-a633-8be2ec822da9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:51:34.194289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d439980-c802-4b37-a633-8be2ec822da9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.195332 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: WARNING py.warnings [None req-3d439980-c802-4b37-a633-8be2ec822da9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 13 21:51:34.195332 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: warnings.warn(msg) May 13 21:51:34.195332 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: May 13 21:51:34.196211 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d439980-c802-4b37-a633-8be2ec822da9 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.197010 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 780/1550] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:34.198360 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae486a5-7184-4cec-93b2-3df54140438d None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.198658 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae486a5-7184-4cec-93b2-3df54140438d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.198963 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae486a5-7184-4cec-93b2-3df54140438d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.209056 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 771/1551] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:51:34.210596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebac1f4-b08c-4db4-890f-4b3af73052da None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.210917 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebac1f4-b08c-4db4-890f-4b3af73052da None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.211196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebac1f4-b08c-4db4-890f-4b3af73052da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.222849 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7ae486a5-7184-4cec-93b2-3df54140438d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:34.224096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7ae486a5-7184-4cec-93b2-3df54140438d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:34.228105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-eebac1f4-b08c-4db4-890f-4b3af73052da None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:34.229263 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-eebac1f4-b08c-4db4-890f-4b3af73052da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:34.232505 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 781/1552] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:34.233267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae486a5-7184-4cec-93b2-3df54140438d None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.238674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 772/1553] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:34.239995 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77125bfa-1da0-405b-a7a7-1f950f40005f None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.240288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77125bfa-1da0-405b-a7a7-1f950f40005f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.240567 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77125bfa-1da0-405b-a7a7-1f950f40005f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.243140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae486a5-7184-4cec-93b2-3df54140438d None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.248203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cb296-ebfc-45b4-9178-44556841402c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.248854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cb296-ebfc-45b4-9178-44556841402c admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.249136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cb296-ebfc-45b4-9178-44556841402c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.249412 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cb296-ebfc-45b4-9178-44556841402c admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.250128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2cb296-ebfc-45b4-9178-44556841402c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:51:34.250432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2cb296-ebfc-45b4-9178-44556841402c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.252088 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2cb296-ebfc-45b4-9178-44556841402c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.258739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-77125bfa-1da0-405b-a7a7-1f950f40005f None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:34.259953 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-77125bfa-1da0-405b-a7a7-1f950f40005f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:34.263702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 782/1554] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:51:34.264327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cb296-ebfc-45b4-9178-44556841402c admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.272872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 773/1555] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:34.274220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0683b-fd48-4899-ae29-d760f6902558 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.274521 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0683b-fd48-4899-ae29-d760f6902558 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.274834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0683b-fd48-4899-ae29-d760f6902558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.278435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cb296-ebfc-45b4-9178-44556841402c admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.283418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73d24b-bb01-4aaf-8ae8-56aead3d8fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.284072 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d24b-bb01-4aaf-8ae8-56aead3d8fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.284356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d24b-bb01-4aaf-8ae8-56aead3d8fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.284633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d24b-bb01-4aaf-8ae8-56aead3d8fd0 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.285313 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73d24b-bb01-4aaf-8ae8-56aead3d8fd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.287326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73d24b-bb01-4aaf-8ae8-56aead3d8fd0 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.292266 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-2fe0683b-fd48-4899-ae29-d760f6902558 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:34.293461 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2fe0683b-fd48-4899-ae29-d760f6902558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:34.302062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 774/1556] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:34.303395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011121a4-3554-4023-90cc-f5ebcec57b2c None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.303635 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011121a4-3554-4023-90cc-f5ebcec57b2c None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.303961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011121a4-3554-4023-90cc-f5ebcec57b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.312435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 783/1557] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/users => generated 282 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:34.312435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73d24b-bb01-4aaf-8ae8-56aead3d8fd0 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.321745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73d24b-bb01-4aaf-8ae8-56aead3d8fd0 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.322157 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-011121a4-3554-4023-90cc-f5ebcec57b2c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:34.323421 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-011121a4-3554-4023-90cc-f5ebcec57b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:34.326305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a232f-f797-4234-945d-91cfe1301d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.326960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a232f-f797-4234-945d-91cfe1301d74 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.327249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a232f-f797-4234-945d-91cfe1301d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.327525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a232f-f797-4234-945d-91cfe1301d74 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.328200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a232f-f797-4234-945d-91cfe1301d74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.330126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a232f-f797-4234-945d-91cfe1301d74 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.331641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 775/1558] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:34.332407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011121a4-3554-4023-90cc-f5ebcec57b2c None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.341719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011121a4-3554-4023-90cc-f5ebcec57b2c None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.346335 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ae6ac-564c-49c8-be69-0b41e7a7ced6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.346987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ae6ac-564c-49c8-be69-0b41e7a7ced6 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.347272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ae6ac-564c-49c8-be69-0b41e7a7ced6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.347542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ae6ac-564c-49c8-be69-0b41e7a7ced6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.348247 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ae6ac-564c-49c8-be69-0b41e7a7ced6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:51:34.348547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ae6ac-564c-49c8-be69-0b41e7a7ced6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.350289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ae6ac-564c-49c8-be69-0b41e7a7ced6 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.363526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 784/1559] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:34.363526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f59436-4d7f-49d5-afae-7880b197f359 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.363526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f59436-4d7f-49d5-afae-7880b197f359 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.363526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f59436-4d7f-49d5-afae-7880b197f359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.366322 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 776/1560] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:51:34.367220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ae6ac-564c-49c8-be69-0b41e7a7ced6 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.381347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ae6ac-564c-49c8-be69-0b41e7a7ced6 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.385135 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-64f59436-4d7f-49d5-afae-7880b197f359 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:34.386328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-64f59436-4d7f-49d5-afae-7880b197f359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:34.436453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b709ec0-2215-45a7-a2d9-d7ec9362ed98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.437135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b709ec0-2215-45a7-a2d9-d7ec9362ed98 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.437421 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b709ec0-2215-45a7-a2d9-d7ec9362ed98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.437697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b709ec0-2215-45a7-a2d9-d7ec9362ed98 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.438379 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b709ec0-2215-45a7-a2d9-d7ec9362ed98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.440378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b709ec0-2215-45a7-a2d9-d7ec9362ed98 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.442285 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 785/1561] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:34.443672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d0741-9c70-47b8-8adf-6f6fed3772d5 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.443965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d0741-9c70-47b8-8adf-6f6fed3772d5 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.444425 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d0741-9c70-47b8-8adf-6f6fed3772d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.461669 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-aa0d0741-9c70-47b8-8adf-6f6fed3772d5 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:34.462895 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-aa0d0741-9c70-47b8-8adf-6f6fed3772d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:34.466356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 777/1562] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:34.467257 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b709ec0-2215-45a7-a2d9-d7ec9362ed98 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.521416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 786/1563] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:34.525567 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b709ec0-2215-45a7-a2d9-d7ec9362ed98 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.525724 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d0741-9c70-47b8-8adf-6f6fed3772d5 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.530134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d7938-7015-466c-88c4-c5a2056cd878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.530819 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d7938-7015-466c-88c4-c5a2056cd878 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.531104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d7938-7015-466c-88c4-c5a2056cd878 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.531378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d7938-7015-466c-88c4-c5a2056cd878 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.532092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d7938-7015-466c-88c4-c5a2056cd878 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:51:34.532401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d7938-7015-466c-88c4-c5a2056cd878 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.533985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d7938-7015-466c-88c4-c5a2056cd878 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.534454 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d0741-9c70-47b8-8adf-6f6fed3772d5 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.538943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594606e4-477c-46fe-b4ed-6dfbfd400119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.539576 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594606e4-477c-46fe-b4ed-6dfbfd400119 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.539854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594606e4-477c-46fe-b4ed-6dfbfd400119 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.540127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594606e4-477c-46fe-b4ed-6dfbfd400119 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.540827 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594606e4-477c-46fe-b4ed-6dfbfd400119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.542398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594606e4-477c-46fe-b4ed-6dfbfd400119 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.545660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 778/1564] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:51:34.546446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d7938-7015-466c-88c4-c5a2056cd878 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.553047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 787/1565] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:34.553757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594606e4-477c-46fe-b4ed-6dfbfd400119 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.559481 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d7938-7015-466c-88c4-c5a2056cd878 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.563504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594606e4-477c-46fe-b4ed-6dfbfd400119 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.564720 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97b465-7aa2-40bd-851f-1cd2420c7070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.565366 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97b465-7aa2-40bd-851f-1cd2420c7070 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.565647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97b465-7aa2-40bd-851f-1cd2420c7070 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.565924 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97b465-7aa2-40bd-851f-1cd2420c7070 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.566633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97b465-7aa2-40bd-851f-1cd2420c7070 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:51:34.566967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97b465-7aa2-40bd-851f-1cd2420c7070 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.568141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf275749-52b0-4e2c-b4b9-4076da62c370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.568546 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97b465-7aa2-40bd-851f-1cd2420c7070 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.568797 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf275749-52b0-4e2c-b4b9-4076da62c370 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.569076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf275749-52b0-4e2c-b4b9-4076da62c370 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.569349 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf275749-52b0-4e2c-b4b9-4076da62c370 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.570018 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf275749-52b0-4e2c-b4b9-4076da62c370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.571946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf275749-52b0-4e2c-b4b9-4076da62c370 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.580218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 779/1566] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:51:34.580978 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97b465-7aa2-40bd-851f-1cd2420c7070 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.590476 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97b465-7aa2-40bd-851f-1cd2420c7070 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.596204 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc84c6e-92b3-4ea9-8253-bc566e980c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.596856 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc84c6e-92b3-4ea9-8253-bc566e980c5a admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.597136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc84c6e-92b3-4ea9-8253-bc566e980c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.597410 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc84c6e-92b3-4ea9-8253-bc566e980c5a admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.598104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc84c6e-92b3-4ea9-8253-bc566e980c5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.601858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 788/1567] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/users => generated 303 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:34.602003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc84c6e-92b3-4ea9-8253-bc566e980c5a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.602131 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf275749-52b0-4e2c-b4b9-4076da62c370 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.610219 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf275749-52b0-4e2c-b4b9-4076da62c370 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.615858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca7506-4fe2-4c10-8f5f-ff7f57057343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.616527 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca7506-4fe2-4c10-8f5f-ff7f57057343 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.617063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca7506-4fe2-4c10-8f5f-ff7f57057343 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.617334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca7506-4fe2-4c10-8f5f-ff7f57057343 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.618103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca7506-4fe2-4c10-8f5f-ff7f57057343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:51:34.618415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca7506-4fe2-4c10-8f5f-ff7f57057343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.620106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca7506-4fe2-4c10-8f5f-ff7f57057343 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.626376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 780/1568] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:34.627195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc84c6e-92b3-4ea9-8253-bc566e980c5a admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.632795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 789/1569] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:51:34.633605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca7506-4fe2-4c10-8f5f-ff7f57057343 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.643062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca7506-4fe2-4c10-8f5f-ff7f57057343 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.692031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc84c6e-92b3-4ea9-8253-bc566e980c5a admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.696590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e1e06-412a-4c4b-a008-14434c0e1606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.697235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85979fb4-fb7c-4d43-9df7-4cb3022bd566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.697362 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e1e06-412a-4c4b-a008-14434c0e1606 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.697520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e1e06-412a-4c4b-a008-14434c0e1606 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.697796 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e1e06-412a-4c4b-a008-14434c0e1606 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.697898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85979fb4-fb7c-4d43-9df7-4cb3022bd566 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.698163 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85979fb4-fb7c-4d43-9df7-4cb3022bd566 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.698443 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85979fb4-fb7c-4d43-9df7-4cb3022bd566 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.698534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e1e06-412a-4c4b-a008-14434c0e1606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.699191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85979fb4-fb7c-4d43-9df7-4cb3022bd566 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:51:34.699504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85979fb4-fb7c-4d43-9df7-4cb3022bd566 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.699621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: WARNING py.warnings [None req-fa2e1e06-412a-4c4b-a008-14434c0e1606 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 21:51:34.699621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: warnings.warn(msg) May 13 21:51:34.699621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: May 13 21:51:34.700751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e1e06-412a-4c4b-a008-14434c0e1606 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.701123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85979fb4-fb7c-4d43-9df7-4cb3022bd566 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.712816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 790/1570] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:51:34.713863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85979fb4-fb7c-4d43-9df7-4cb3022bd566 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.721653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 781/1571] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:34.722481 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e1e06-412a-4c4b-a008-14434c0e1606 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.724426 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85979fb4-fb7c-4d43-9df7-4cb3022bd566 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.729117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a31e5-7370-4ddd-b725-f7e1641fc7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.729764 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a31e5-7370-4ddd-b725-f7e1641fc7ad admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.730040 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a31e5-7370-4ddd-b725-f7e1641fc7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.730313 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a31e5-7370-4ddd-b725-f7e1641fc7ad admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.731028 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a31e5-7370-4ddd-b725-f7e1641fc7ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.732896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e1e06-412a-4c4b-a008-14434c0e1606 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.733740 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a31e5-7370-4ddd-b725-f7e1641fc7ad admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.737908 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce768f7-a520-4edc-ad1d-2f4eed05be68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.738550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce768f7-a520-4edc-ad1d-2f4eed05be68 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.738878 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce768f7-a520-4edc-ad1d-2f4eed05be68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.739157 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce768f7-a520-4edc-ad1d-2f4eed05be68 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.739931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce768f7-a520-4edc-ad1d-2f4eed05be68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.741549 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce768f7-a520-4edc-ad1d-2f4eed05be68 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.755929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 782/1572] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:34.755929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce768f7-a520-4edc-ad1d-2f4eed05be68 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.759951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 791/1573] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:34.760999 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a31e5-7370-4ddd-b725-f7e1641fc7ad admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.767099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce768f7-a520-4edc-ad1d-2f4eed05be68 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.822223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7814fd8-bc77-4255-901f-a0af344d056a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.822820 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7814fd8-bc77-4255-901f-a0af344d056a admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.823104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7814fd8-bc77-4255-901f-a0af344d056a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.823379 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7814fd8-bc77-4255-901f-a0af344d056a admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.824057 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7814fd8-bc77-4255-901f-a0af344d056a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.825979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7814fd8-bc77-4255-901f-a0af344d056a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.826831 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a31e5-7370-4ddd-b725-f7e1641fc7ad admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.831463 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19f2ff-d057-47d6-bb22-12d9a818e81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.832123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19f2ff-d057-47d6-bb22-12d9a818e81a admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.832408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19f2ff-d057-47d6-bb22-12d9a818e81a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.832694 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19f2ff-d057-47d6-bb22-12d9a818e81a admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.833408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19f2ff-d057-47d6-bb22-12d9a818e81a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.835112 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19f2ff-d057-47d6-bb22-12d9a818e81a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.845673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 792/1574] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:34.846435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19f2ff-d057-47d6-bb22-12d9a818e81a admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.854604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 783/1575] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/projects => generated 391 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:34.854604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7814fd8-bc77-4255-901f-a0af344d056a admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.855758 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19f2ff-d057-47d6-bb22-12d9a818e81a admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.910386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5259d0a-4a54-4cb7-8bf9-5d4785b4f99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.911079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5259d0a-4a54-4cb7-8bf9-5d4785b4f99e admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.911366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5259d0a-4a54-4cb7-8bf9-5d4785b4f99e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.911645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5259d0a-4a54-4cb7-8bf9-5d4785b4f99e admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.912328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5259d0a-4a54-4cb7-8bf9-5d4785b4f99e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.912930 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7814fd8-bc77-4255-901f-a0af344d056a admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.914283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5259d0a-4a54-4cb7-8bf9-5d4785b4f99e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.917526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2408b-9962-4d57-ba36-92c0f266fb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:34.918188 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2408b-9962-4d57-ba36-92c0f266fb39 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:34.918477 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2408b-9962-4d57-ba36-92c0f266fb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:34.918790 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2408b-9962-4d57-ba36-92c0f266fb39 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:34.920057 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2408b-9962-4d57-ba36-92c0f266fb39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:34.921974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2408b-9962-4d57-ba36-92c0f266fb39 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:34.934425 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 793/1576] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/users => generated 293 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:34.935336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5259d0a-4a54-4cb7-8bf9-5d4785b4f99e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:34.946661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5259d0a-4a54-4cb7-8bf9-5d4785b4f99e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:34.946838 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 784/1577] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/projects => generated 403 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:34.951545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2408b-9962-4d57-ba36-92c0f266fb39 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.001802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0760d9-fabc-4446-a34a-a9869f0795b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.002617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0760d9-fabc-4446-a34a-a9869f0795b2 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.002935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0760d9-fabc-4446-a34a-a9869f0795b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.003215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0760d9-fabc-4446-a34a-a9869f0795b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.003926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0760d9-fabc-4446-a34a-a9869f0795b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.005873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0760d9-fabc-4446-a34a-a9869f0795b2 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.009048 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2408b-9962-4d57-ba36-92c0f266fb39 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.014314 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b109cac-dad9-4e41-886e-32f99b8d89d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.015022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b109cac-dad9-4e41-886e-32f99b8d89d2 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.015334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b109cac-dad9-4e41-886e-32f99b8d89d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.015614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b109cac-dad9-4e41-886e-32f99b8d89d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.016341 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b109cac-dad9-4e41-886e-32f99b8d89d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.017981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b109cac-dad9-4e41-886e-32f99b8d89d2 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.028155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 785/1578] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:35.029489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b109cac-dad9-4e41-886e-32f99b8d89d2 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.035038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 794/1579] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:34 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:35.035038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0760d9-fabc-4446-a34a-a9869f0795b2 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.039071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b109cac-dad9-4e41-886e-32f99b8d89d2 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.093395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d86489-ce22-424d-9ced-b314fb084c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.094179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d86489-ce22-424d-9ced-b314fb084c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.094463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d86489-ce22-424d-9ced-b314fb084c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.094765 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d86489-ce22-424d-9ced-b314fb084c25 admin admin] PATH_INFO: `/v3/projects/79bcd2be37fc4969b1af6424024343aa/users/aa7cac0c3ed34d46ba359f89ee8f68c1/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.095873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0760d9-fabc-4446-a34a-a9869f0795b2 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.100732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afe24d-86a1-41ba-a03b-f37254ac59c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.101449 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afe24d-86a1-41ba-a03b-f37254ac59c9 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.101764 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afe24d-86a1-41ba-a03b-f37254ac59c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.102081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afe24d-86a1-41ba-a03b-f37254ac59c9 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.102804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afe24d-86a1-41ba-a03b-f37254ac59c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.104788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afe24d-86a1-41ba-a03b-f37254ac59c9 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.108778 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d86489-ce22-424d-9ced-b314fb084c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79bcd2be37fc4969b1af6424024343aa, user_id=aa7cac0c3ed34d46ba359f89ee8f68c1, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.111479 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d86489-ce22-424d-9ced-b314fb084c25 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.124614 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 795/1580] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:35 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:35.135096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 786/1581] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:35 2022] PUT /identity/v3/projects/79bcd2be37fc4969b1af6424024343aa/users/aa7cac0c3ed34d46ba359f89ee8f68c1/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:35.135283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afe24d-86a1-41ba-a03b-f37254ac59c9 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.135418 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d86489-ce22-424d-9ced-b314fb084c25 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.144770 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afe24d-86a1-41ba-a03b-f37254ac59c9 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.190110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d86489-ce22-424d-9ced-b314fb084c25 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.195027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cc3e3-6efb-4e9c-ba26-f423c092defa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.195741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cc3e3-6efb-4e9c-ba26-f423c092defa admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.196021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cc3e3-6efb-4e9c-ba26-f423c092defa admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.196292 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cc3e3-6efb-4e9c-ba26-f423c092defa admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.197001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cc3e3-6efb-4e9c-ba26-f423c092defa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.198942 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cc3e3-6efb-4e9c-ba26-f423c092defa admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.199869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384681a0-eeab-4ec2-9a83-f8dfa58ac4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.201036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384681a0-eeab-4ec2-9a83-f8dfa58ac4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.201315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384681a0-eeab-4ec2-9a83-f8dfa58ac4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.201588 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384681a0-eeab-4ec2-9a83-f8dfa58ac4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.202304 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384681a0-eeab-4ec2-9a83-f8dfa58ac4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.203957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384681a0-eeab-4ec2-9a83-f8dfa58ac4a2 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.214491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 796/1582] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:35.218404 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384681a0-eeab-4ec2-9a83-f8dfa58ac4a2 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.222258 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 787/1583] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:35 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:35.222258 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cc3e3-6efb-4e9c-ba26-f423c092defa admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.226053 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384681a0-eeab-4ec2-9a83-f8dfa58ac4a2 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.229533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cc3e3-6efb-4e9c-ba26-f423c092defa admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.230533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c068f9-6908-45e1-8d05-ded3cf914014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.231195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c068f9-6908-45e1-8d05-ded3cf914014 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.231521 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c068f9-6908-45e1-8d05-ded3cf914014 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.231798 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c068f9-6908-45e1-8d05-ded3cf914014 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.232500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c068f9-6908-45e1-8d05-ded3cf914014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.234067 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c068f9-6908-45e1-8d05-ded3cf914014 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.234232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604a574-9c98-49c7-911a-c7fc808861d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.234907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604a574-9c98-49c7-911a-c7fc808861d2 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.235200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604a574-9c98-49c7-911a-c7fc808861d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.235548 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604a574-9c98-49c7-911a-c7fc808861d2 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.236225 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604a574-9c98-49c7-911a-c7fc808861d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.238115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604a574-9c98-49c7-911a-c7fc808861d2 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.244235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 797/1584] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:35.245213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c068f9-6908-45e1-8d05-ded3cf914014 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.254812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c068f9-6908-45e1-8d05-ded3cf914014 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.259330 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f6fc1-1d3b-46db-9a96-eb175c23f874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.263294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 788/1585] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:35 2022] POST /identity/v3/users => generated 302 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:35.263456 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f6fc1-1d3b-46db-9a96-eb175c23f874 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.263456 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f6fc1-1d3b-46db-9a96-eb175c23f874 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.263456 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f6fc1-1d3b-46db-9a96-eb175c23f874 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.263699 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604a574-9c98-49c7-911a-c7fc808861d2 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.263798 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f6fc1-1d3b-46db-9a96-eb175c23f874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.263798 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f6fc1-1d3b-46db-9a96-eb175c23f874 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.270511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604a574-9c98-49c7-911a-c7fc808861d2 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.277246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 798/1586] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:35.277246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f6fc1-1d3b-46db-9a96-eb175c23f874 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.278167 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cce0ee-9c02-4b19-bdd5-29745b665daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.279372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cce0ee-9c02-4b19-bdd5-29745b665daf admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.279654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cce0ee-9c02-4b19-bdd5-29745b665daf admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.279925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cce0ee-9c02-4b19-bdd5-29745b665daf admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.280597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cce0ee-9c02-4b19-bdd5-29745b665daf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.282513 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cce0ee-9c02-4b19-bdd5-29745b665daf admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.287072 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f6fc1-1d3b-46db-9a96-eb175c23f874 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.291571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a80d13-a1f8-4db7-8343-8a4afba6fa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.292194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a80d13-a1f8-4db7-8343-8a4afba6fa7a admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.292477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a80d13-a1f8-4db7-8343-8a4afba6fa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.292815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a80d13-a1f8-4db7-8343-8a4afba6fa7a admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.293765 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a80d13-a1f8-4db7-8343-8a4afba6fa7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.295938 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a80d13-a1f8-4db7-8343-8a4afba6fa7a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.302639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 789/1587] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:35 2022] POST /identity/v3/users => generated 286 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:35.306010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea725f-6216-43c8-b7f7-56d92b101624 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.306010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea725f-6216-43c8-b7f7-56d92b101624 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.306010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea725f-6216-43c8-b7f7-56d92b101624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.306491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 799/1588] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:35.307295 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a80d13-a1f8-4db7-8343-8a4afba6fa7a admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.316688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a80d13-a1f8-4db7-8343-8a4afba6fa7a admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.321288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8562da-55dc-4960-b6c6-e99809710ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.322150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8562da-55dc-4960-b6c6-e99809710ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.322503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8562da-55dc-4960-b6c6-e99809710ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.322807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8562da-55dc-4960-b6c6-e99809710ef8 admin admin] PATH_INFO: `/v3/projects/26f61a0463f04099abea79ab789f2fc9/users/a089cef485964a0b9483391ba5ec3037/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.331680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-46ea725f-6216-43c8-b7f7-56d92b101624 None None] MFA Rules not processed for user `aa7cac0c3ed34d46ba359f89ee8f68c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:35.332987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-46ea725f-6216-43c8-b7f7-56d92b101624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:35.337271 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8562da-55dc-4960-b6c6-e99809710ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26f61a0463f04099abea79ab789f2fc9, user_id=a089cef485964a0b9483391ba5ec3037, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.338313 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: WARNING py.warnings [None req-8c8562da-55dc-4960-b6c6-e99809710ef8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 21:51:35.338313 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: warnings.warn(msg) May 13 21:51:35.338313 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: May 13 21:51:35.340279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8562da-55dc-4960-b6c6-e99809710ef8 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.353939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 800/1589] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:35 2022] PUT /identity/v3/projects/26f61a0463f04099abea79ab789f2fc9/users/a089cef485964a0b9483391ba5ec3037/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:35.354619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8562da-55dc-4960-b6c6-e99809710ef8 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.396805 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 790/1590] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:35 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:35.397668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea725f-6216-43c8-b7f7-56d92b101624 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.408156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea725f-6216-43c8-b7f7-56d92b101624 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.412823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1370e-365e-4bf5-88d9-2579646ca4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.413549 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1370e-365e-4bf5-88d9-2579646ca4ea admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.413835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1370e-365e-4bf5-88d9-2579646ca4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.414107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1370e-365e-4bf5-88d9-2579646ca4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.414857 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1370e-365e-4bf5-88d9-2579646ca4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.414967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8562da-55dc-4960-b6c6-e99809710ef8 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.416586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1370e-365e-4bf5-88d9-2579646ca4ea admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.419615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd86f3-dca7-424b-af23-99966e14140d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.420273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd86f3-dca7-424b-af23-99966e14140d admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.420557 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd86f3-dca7-424b-af23-99966e14140d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.420844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd86f3-dca7-424b-af23-99966e14140d admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.421559 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd86f3-dca7-424b-af23-99966e14140d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.423193 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd86f3-dca7-424b-af23-99966e14140d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.426328 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 791/1591] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:35.427140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1370e-365e-4bf5-88d9-2579646ca4ea admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.436441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 801/1592] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:35.436441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd86f3-dca7-424b-af23-99966e14140d admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.436709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1370e-365e-4bf5-88d9-2579646ca4ea admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.441298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec9dd8-6703-4eb8-b117-c1c3697a5609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.441924 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec9dd8-6703-4eb8-b117-c1c3697a5609 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.442213 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec9dd8-6703-4eb8-b117-c1c3697a5609 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.442490 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec9dd8-6703-4eb8-b117-c1c3697a5609 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.443227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec9dd8-6703-4eb8-b117-c1c3697a5609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.444789 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec9dd8-6703-4eb8-b117-c1c3697a5609 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.445979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd86f3-dca7-424b-af23-99966e14140d admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.451115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ca206-be82-421f-9b14-9c417ff00c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.451992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ca206-be82-421f-9b14-9c417ff00c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.452266 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ca206-be82-421f-9b14-9c417ff00c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.452533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ca206-be82-421f-9b14-9c417ff00c36 admin admin] PATH_INFO: `/v3/projects/6a12e6fa2119499b93806f036eb55a0c/users/0a1425d092844a2e9b8a917dc9981b32/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.455245 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 792/1593] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:35.455879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec9dd8-6703-4eb8-b117-c1c3697a5609 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.465304 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec9dd8-6703-4eb8-b117-c1c3697a5609 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.467232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ca206-be82-421f-9b14-9c417ff00c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a12e6fa2119499b93806f036eb55a0c, user_id=0a1425d092844a2e9b8a917dc9981b32, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.470059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43da8f7-2557-4051-a705-06565d2da480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.470214 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ca206-be82-421f-9b14-9c417ff00c36 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.470871 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43da8f7-2557-4051-a705-06565d2da480 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.471161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43da8f7-2557-4051-a705-06565d2da480 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.471440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43da8f7-2557-4051-a705-06565d2da480 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.472159 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43da8f7-2557-4051-a705-06565d2da480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.473821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43da8f7-2557-4051-a705-06565d2da480 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.487246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 802/1594] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:35 2022] PUT /identity/v3/projects/6a12e6fa2119499b93806f036eb55a0c/users/0a1425d092844a2e9b8a917dc9981b32/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:35.487246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ca206-be82-421f-9b14-9c417ff00c36 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.487544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 793/1595] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:35.487544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f7265-0d88-44a3-83ac-b5eef395b01d None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.487544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f7265-0d88-44a3-83ac-b5eef395b01d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.487841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f7265-0d88-44a3-83ac-b5eef395b01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.514575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-6f6f7265-0d88-44a3-83ac-b5eef395b01d None None] MFA Rules not processed for user `a089cef485964a0b9483391ba5ec3037`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:35.515902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-6f6f7265-0d88-44a3-83ac-b5eef395b01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:35.546274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ca206-be82-421f-9b14-9c417ff00c36 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.550829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf2144-c9fa-4b6e-98bd-07630b68538c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.551480 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf2144-c9fa-4b6e-98bd-07630b68538c admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.551759 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf2144-c9fa-4b6e-98bd-07630b68538c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.552030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf2144-c9fa-4b6e-98bd-07630b68538c admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.552732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf2144-c9fa-4b6e-98bd-07630b68538c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.554328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf2144-c9fa-4b6e-98bd-07630b68538c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.564641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 803/1596] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:35.565674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf2144-c9fa-4b6e-98bd-07630b68538c admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.576220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf2144-c9fa-4b6e-98bd-07630b68538c admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.576484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 794/1597] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:35 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:35.577281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f7265-0d88-44a3-83ac-b5eef395b01d None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.581499 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ee70e-f69e-4711-96f9-d88fc299f7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.582133 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ee70e-f69e-4711-96f9-d88fc299f7f7 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.582427 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ee70e-f69e-4711-96f9-d88fc299f7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.582714 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ee70e-f69e-4711-96f9-d88fc299f7f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.583405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ee70e-f69e-4711-96f9-d88fc299f7f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.585301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ee70e-f69e-4711-96f9-d88fc299f7f7 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.587034 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f7265-0d88-44a3-83ac-b5eef395b01d None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.591556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fd58b-f96d-4a56-ae1f-ef33ecbf9383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.592408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fd58b-f96d-4a56-ae1f-ef33ecbf9383 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.592686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fd58b-f96d-4a56-ae1f-ef33ecbf9383 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.592958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fd58b-f96d-4a56-ae1f-ef33ecbf9383 admin admin] PATH_INFO: `/v3/projects/cd76838c6c91427f8b0331f5338afeff/users/e456da1e33c14bafa22f975abfc905bc/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.607303 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347fd58b-f96d-4a56-ae1f-ef33ecbf9383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd76838c6c91427f8b0331f5338afeff, user_id=e456da1e33c14bafa22f975abfc905bc, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.609908 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347fd58b-f96d-4a56-ae1f-ef33ecbf9383 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.610594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 804/1598] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:35 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:35.611839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ee70e-f69e-4711-96f9-d88fc299f7f7 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.625123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 795/1599] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:35 2022] PUT /identity/v3/projects/cd76838c6c91427f8b0331f5338afeff/users/e456da1e33c14bafa22f975abfc905bc/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:35.626071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fd58b-f96d-4a56-ae1f-ef33ecbf9383 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.674630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ee70e-f69e-4711-96f9-d88fc299f7f7 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.684528 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fd58b-f96d-4a56-ae1f-ef33ecbf9383 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.689858 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f1678-0344-4682-ab37-5d78035d2756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.690547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f1678-0344-4682-ab37-5d78035d2756 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.690913 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f1678-0344-4682-ab37-5d78035d2756 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.691199 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f1678-0344-4682-ab37-5d78035d2756 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.691923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f1678-0344-4682-ab37-5d78035d2756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.693518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f1678-0344-4682-ab37-5d78035d2756 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.706898 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 796/1600] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:35.706898 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a84580-1f5f-44e7-b8ce-cf3ef4db0ccc None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.706898 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a84580-1f5f-44e7-b8ce-cf3ef4db0ccc None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.706898 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a84580-1f5f-44e7-b8ce-cf3ef4db0ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.722135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-61a84580-1f5f-44e7-b8ce-cf3ef4db0ccc None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:35.723472 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-61a84580-1f5f-44e7-b8ce-cf3ef4db0ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:35.728294 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e01859-820c-4e16-8e32-b7f23ca22bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.728957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e01859-820c-4e16-8e32-b7f23ca22bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.729238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e01859-820c-4e16-8e32-b7f23ca22bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.729512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e01859-820c-4e16-8e32-b7f23ca22bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.730223 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e01859-820c-4e16-8e32-b7f23ca22bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.731088 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 797/1601] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:35.732431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a84580-1f5f-44e7-b8ce-cf3ef4db0ccc None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.732549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e01859-820c-4e16-8e32-b7f23ca22bf9 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.742948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 805/1602] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:35.743739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e01859-820c-4e16-8e32-b7f23ca22bf9 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.746771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a84580-1f5f-44e7-b8ce-cf3ef4db0ccc None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.751994 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad8e1c-2ab1-47dd-b7a6-4f802d5226c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.752652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad8e1c-2ab1-47dd-b7a6-4f802d5226c4 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.752967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad8e1c-2ab1-47dd-b7a6-4f802d5226c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.753278 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad8e1c-2ab1-47dd-b7a6-4f802d5226c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.753466 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e01859-820c-4e16-8e32-b7f23ca22bf9 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.754004 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad8e1c-2ab1-47dd-b7a6-4f802d5226c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.755649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad8e1c-2ab1-47dd-b7a6-4f802d5226c4 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.758795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d02417-4b7b-47c3-893d-64aa9b8ff9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.759644 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d02417-4b7b-47c3-893d-64aa9b8ff9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.759923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d02417-4b7b-47c3-893d-64aa9b8ff9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.760194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d02417-4b7b-47c3-893d-64aa9b8ff9bf admin admin] PATH_INFO: `/v3/projects/39174f764455497c99b2fc47838407f8/users/1315f98467d8426094229e3a31c618f3/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.765478 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 798/1603] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:35.766315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad8e1c-2ab1-47dd-b7a6-4f802d5226c4 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.774561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d02417-4b7b-47c3-893d-64aa9b8ff9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=39174f764455497c99b2fc47838407f8, user_id=1315f98467d8426094229e3a31c618f3, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.775799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad8e1c-2ab1-47dd-b7a6-4f802d5226c4 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.777423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d02417-4b7b-47c3-893d-64aa9b8ff9bf admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.780601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bb8d2-5e7c-4ac5-8649-c4574398ee52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.781226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bb8d2-5e7c-4ac5-8649-c4574398ee52 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.781503 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bb8d2-5e7c-4ac5-8649-c4574398ee52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.782058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bb8d2-5e7c-4ac5-8649-c4574398ee52 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.783175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384bb8d2-5e7c-4ac5-8649-c4574398ee52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.785447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384bb8d2-5e7c-4ac5-8649-c4574398ee52 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.790968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 806/1604] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:35 2022] PUT /identity/v3/projects/39174f764455497c99b2fc47838407f8/users/1315f98467d8426094229e3a31c618f3/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:35.791740 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d02417-4b7b-47c3-893d-64aa9b8ff9bf admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.815528 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 799/1605] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:35 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:35.816783 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9d5aa-1ac8-4f37-b861-ef2cef75f580 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.817071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9d5aa-1ac8-4f37-b861-ef2cef75f580 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.817347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9d5aa-1ac8-4f37-b861-ef2cef75f580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.844552 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-b2c9d5aa-1ac8-4f37-b861-ef2cef75f580 None None] MFA Rules not processed for user `e456da1e33c14bafa22f975abfc905bc`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:35.845761 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b2c9d5aa-1ac8-4f37-b861-ef2cef75f580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:35.852468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d02417-4b7b-47c3-893d-64aa9b8ff9bf admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.907669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 800/1606] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:35.908204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7796f13-a581-47ad-b410-3fa8fbd02d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.908386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9d5aa-1ac8-4f37-b861-ef2cef75f580 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.908702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7796f13-a581-47ad-b410-3fa8fbd02d31 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.909064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7796f13-a581-47ad-b410-3fa8fbd02d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.909349 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7796f13-a581-47ad-b410-3fa8fbd02d31 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.910036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7796f13-a581-47ad-b410-3fa8fbd02d31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.912178 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7796f13-a581-47ad-b410-3fa8fbd02d31 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.922298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9d5aa-1ac8-4f37-b861-ef2cef75f580 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:35.928328 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a30ab2-3550-4172-9648-1ee0d9dfa3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:35.929220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a30ab2-3550-4172-9648-1ee0d9dfa3a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:35.929517 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a30ab2-3550-4172-9648-1ee0d9dfa3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:35.929792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a30ab2-3550-4172-9648-1ee0d9dfa3a9 admin admin] PATH_INFO: `/v3/projects/5b7079f32018466e99a74b43dbb0a5f8/users/271827a6cf9344318a279d4daf6e4a09/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:35.939799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 807/1607] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:35 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:35.939799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7796f13-a581-47ad-b410-3fa8fbd02d31 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:35.947672 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a30ab2-3550-4172-9648-1ee0d9dfa3a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7079f32018466e99a74b43dbb0a5f8, user_id=271827a6cf9344318a279d4daf6e4a09, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:35.950368 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a30ab2-3550-4172-9648-1ee0d9dfa3a9 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:35.971226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 801/1608] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:35 2022] PUT /identity/v3/projects/5b7079f32018466e99a74b43dbb0a5f8/users/271827a6cf9344318a279d4daf6e4a09/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:35.972360 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a30ab2-3550-4172-9648-1ee0d9dfa3a9 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.000729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7796f13-a581-47ad-b410-3fa8fbd02d31 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:36.032476 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a30ab2-3550-4172-9648-1ee0d9dfa3a9 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:36.037218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aee17f-bc9a-4819-b4d7-7da1ffb45729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:36.038104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aee17f-bc9a-4819-b4d7-7da1ffb45729 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.038387 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aee17f-bc9a-4819-b4d7-7da1ffb45729 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.038662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aee17f-bc9a-4819-b4d7-7da1ffb45729 admin admin] PATH_INFO: `/v3/projects/428d047f511945e5bdf35eb8ed236507/users/6b11a3de8b71482594a04ab7e53a94bd/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.047007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250fc7b-e85b-42a3-902f-c6815f50278b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:36.047643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250fc7b-e85b-42a3-902f-c6815f50278b service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.047922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250fc7b-e85b-42a3-902f-c6815f50278b service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.048199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250fc7b-e85b-42a3-902f-c6815f50278b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.052970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aee17f-bc9a-4819-b4d7-7da1ffb45729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=428d047f511945e5bdf35eb8ed236507, user_id=6b11a3de8b71482594a04ab7e53a94bd, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:36.055673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aee17f-bc9a-4819-b4d7-7da1ffb45729 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:36.056926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250fc7b-e85b-42a3-902f-c6815f50278b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:36.058357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250fc7b-e85b-42a3-902f-c6815f50278b service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:36.068292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 808/1609] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:36.072451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 802/1610] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:35 2022] PUT /identity/v3/projects/428d047f511945e5bdf35eb8ed236507/users/6b11a3de8b71482594a04ab7e53a94bd/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:36.072608 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250fc7b-e85b-42a3-902f-c6815f50278b service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.072705 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d1a9e-6523-4f92-88a7-5da95a300e3c None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.072705 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d1a9e-6523-4f92-88a7-5da95a300e3c None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.072705 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d1a9e-6523-4f92-88a7-5da95a300e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.078210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250fc7b-e85b-42a3-902f-c6815f50278b service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:36.089725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-9f3d1a9e-6523-4f92-88a7-5da95a300e3c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:36.092173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-9f3d1a9e-6523-4f92-88a7-5da95a300e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:36.144881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f787429-78e9-48f2-88d0-343e93014958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:36.146165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f787429-78e9-48f2-88d0-343e93014958 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.146625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f787429-78e9-48f2-88d0-343e93014958 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.147114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f787429-78e9-48f2-88d0-343e93014958 admin admin] PATH_INFO: `/v3/projects/51ed5e55a27c4adca6394a55b5cbd7aa/users/ff694cc1ed2d4b99a2dbfaeffcb071d0/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.163647 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f787429-78e9-48f2-88d0-343e93014958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ed5e55a27c4adca6394a55b5cbd7aa, user_id=ff694cc1ed2d4b99a2dbfaeffcb071d0, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:36.164616 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 803/1611] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:36.165919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d1a9e-6523-4f92-88a7-5da95a300e3c None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.166693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f787429-78e9-48f2-88d0-343e93014958 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:36.178621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d1a9e-6523-4f92-88a7-5da95a300e3c None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:36.182675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 809/1612] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:36 2022] PUT /identity/v3/projects/51ed5e55a27c4adca6394a55b5cbd7aa/users/ff694cc1ed2d4b99a2dbfaeffcb071d0/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:36.184203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77585141-c5d6-4c20-bc9b-56d288f8a913 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.184491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77585141-c5d6-4c20-bc9b-56d288f8a913 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.184803 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77585141-c5d6-4c20-bc9b-56d288f8a913 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.186235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 810/1613] 10.222.0.35 () {54 vars in 961 bytes} [Fri May 13 21:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:51:36.187037 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77585141-c5d6-4c20-bc9b-56d288f8a913 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.245236 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31cc1bc-d41b-4135-8323-bf1f900ddbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:36.245876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cc1bc-d41b-4135-8323-bf1f900ddbc7 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.246201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cc1bc-d41b-4135-8323-bf1f900ddbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.246373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cc1bc-d41b-4135-8323-bf1f900ddbc7 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.247267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31cc1bc-d41b-4135-8323-bf1f900ddbc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:36.251569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31cc1bc-d41b-4135-8323-bf1f900ddbc7 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:36.263189 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77585141-c5d6-4c20-bc9b-56d288f8a913 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:36.269563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eecc89-45c9-4da7-a688-db78c21635b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:36.270211 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eecc89-45c9-4da7-a688-db78c21635b3 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.271136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eecc89-45c9-4da7-a688-db78c21635b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.271480 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eecc89-45c9-4da7-a688-db78c21635b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.272157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eecc89-45c9-4da7-a688-db78c21635b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:36.273779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eecc89-45c9-4da7-a688-db78c21635b3 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:36.275119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 804/1614] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:36 2022] POST /identity/v3/users => generated 308 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:36.277237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b4e93-fba8-4651-9595-34c0b7456f39 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.277792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b4e93-fba8-4651-9595-34c0b7456f39 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.278284 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b4e93-fba8-4651-9595-34c0b7456f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.287793 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 811/1615] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:36.289430 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8597c30-9933-4e09-87bb-1be86bd4746b None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.289885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8597c30-9933-4e09-87bb-1be86bd4746b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.290165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8597c30-9933-4e09-87bb-1be86bd4746b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.299360 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-a29b4e93-fba8-4651-9595-34c0b7456f39 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:36.300833 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a29b4e93-fba8-4651-9595-34c0b7456f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:36.309624 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-f8597c30-9933-4e09-87bb-1be86bd4746b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:36.310595 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f8597c30-9933-4e09-87bb-1be86bd4746b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:36.313714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 805/1616] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:36.314923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b4e93-fba8-4651-9595-34c0b7456f39 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.332140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 812/1617] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:36.332363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dc49d-2296-4c34-ae9b-2f9e4739de84 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.332606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dc49d-2296-4c34-ae9b-2f9e4739de84 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.332765 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dc49d-2296-4c34-ae9b-2f9e4739de84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.362103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-795dc49d-2296-4c34-ae9b-2f9e4739de84 None None] MFA Rules not processed for user `ff694cc1ed2d4b99a2dbfaeffcb071d0`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:36.363341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-795dc49d-2296-4c34-ae9b-2f9e4739de84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:36.385924 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b4e93-fba8-4651-9595-34c0b7456f39 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:36.390629 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03a861-5c1c-467c-a9d2-d06411a7cde4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:36.391607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03a861-5c1c-467c-a9d2-d06411a7cde4 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.391891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03a861-5c1c-467c-a9d2-d06411a7cde4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.392166 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03a861-5c1c-467c-a9d2-d06411a7cde4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.402642 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03a861-5c1c-467c-a9d2-d06411a7cde4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:36.405801 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03a861-5c1c-467c-a9d2-d06411a7cde4 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:36.416303 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 806/1618] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:36.422935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0c01-402c-4d34-b411-8ee2564e936e None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.423769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0c01-402c-4d34-b411-8ee2564e936e None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.424346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0c01-402c-4d34-b411-8ee2564e936e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.435922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 813/1619] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:36.437060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795dc49d-2296-4c34-ae9b-2f9e4739de84 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.448288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795dc49d-2296-4c34-ae9b-2f9e4739de84 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:36.450878 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-6a8b0c01-402c-4d34-b411-8ee2564e936e None None] MFA Rules not processed for user `316ef43c6c304bd3a27e085bf4da8a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:36.452469 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-6a8b0c01-402c-4d34-b411-8ee2564e936e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:36.454959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f91c5-9366-46b4-b9af-ae4802cf0d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:36.455860 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f91c5-9366-46b4-b9af-ae4802cf0d86 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.456305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f91c5-9366-46b4-b9af-ae4802cf0d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.456772 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f91c5-9366-46b4-b9af-ae4802cf0d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.457655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f91c5-9366-46b4-b9af-ae4802cf0d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:36.459472 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f91c5-9366-46b4-b9af-ae4802cf0d86 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:36.475100 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 814/1620] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:36.476145 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f91c5-9366-46b4-b9af-ae4802cf0d86 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.488120 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f91c5-9366-46b4-b9af-ae4802cf0d86 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:36.493919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38c4dc-9e2f-4195-9796-6a321c78ae0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:36.495175 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38c4dc-9e2f-4195-9796-6a321c78ae0d admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.495515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38c4dc-9e2f-4195-9796-6a321c78ae0d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.495780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38c4dc-9e2f-4195-9796-6a321c78ae0d admin admin] PATH_INFO: `/v3/projects/f14a4240394542feb3e5391dd674f64c/users/ab066c29a1044cb09bb850dcc59e2a6e/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.511785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38c4dc-9e2f-4195-9796-6a321c78ae0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14a4240394542feb3e5391dd674f64c, user_id=ab066c29a1044cb09bb850dcc59e2a6e, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:36.515532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38c4dc-9e2f-4195-9796-6a321c78ae0d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:36.535815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 815/1621] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:36 2022] PUT /identity/v3/projects/f14a4240394542feb3e5391dd674f64c/users/ab066c29a1044cb09bb850dcc59e2a6e/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:36.535815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38c4dc-9e2f-4195-9796-6a321c78ae0d admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.540658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 807/1622] 10.222.0.35 () {58 vars in 1058 bytes} [Fri May 13 21:51:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:36.541695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b0c01-402c-4d34-b411-8ee2564e936e None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.550536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b0c01-402c-4d34-b411-8ee2564e936e None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:36.600147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38c4dc-9e2f-4195-9796-6a321c78ae0d admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:36.605057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc92e7d-ae12-4ae7-9f63-a740479caec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:36.605754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aa190-b33b-44ca-841a-f709b25d6268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:36.606023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc92e7d-ae12-4ae7-9f63-a740479caec9 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.606407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa190-b33b-44ca-841a-f709b25d6268 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.606896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa190-b33b-44ca-841a-f709b25d6268 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.607068 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc92e7d-ae12-4ae7-9f63-a740479caec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.607164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa190-b33b-44ca-841a-f709b25d6268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.607514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc92e7d-ae12-4ae7-9f63-a740479caec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.672111 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9aa190-b33b-44ca-841a-f709b25d6268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:36.673739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9aa190-b33b-44ca-841a-f709b25d6268 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:36.675701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc92e7d-ae12-4ae7-9f63-a740479caec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:36.677131 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc92e7d-ae12-4ae7-9f63-a740479caec9 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:36.684549 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 808/1623] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:36.685878 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ead2e-7306-42c8-897d-9843e310f828 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.686287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ead2e-7306-42c8-897d-9843e310f828 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.686662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ead2e-7306-42c8-897d-9843e310f828 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.688109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 809/1624] 10.222.0.35 () {54 vars in 961 bytes} [Fri May 13 21:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:51:36.691876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ead2e-7306-42c8-897d-9843e310f828 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.692173 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 816/1625] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:36.692994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc92e7d-ae12-4ae7-9f63-a740479caec9 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.702896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc92e7d-ae12-4ae7-9f63-a740479caec9 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:36.707672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7c6bb-00cc-49de-a17f-482af01b8bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:36.708269 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ead2e-7306-42c8-897d-9843e310f828 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:36.708396 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7c6bb-00cc-49de-a17f-482af01b8bfe admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.708595 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7c6bb-00cc-49de-a17f-482af01b8bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.709189 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7c6bb-00cc-49de-a17f-482af01b8bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.709577 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7c6bb-00cc-49de-a17f-482af01b8bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:36.711366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7c6bb-00cc-49de-a17f-482af01b8bfe admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:36.715152 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92d818-fc6b-41f0-ab5f-4ef84136e16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:36.716780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92d818-fc6b-41f0-ab5f-4ef84136e16b admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.717254 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92d818-fc6b-41f0-ab5f-4ef84136e16b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.717553 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92d818-fc6b-41f0-ab5f-4ef84136e16b admin admin] PATH_INFO: `/v3/projects/8813709134a74a7898f5b4cc8fda4544/users/563d043ab1ed442b98931bb3c83a5c62/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.727935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 817/1626] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:36.727935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7c6bb-00cc-49de-a17f-482af01b8bfe admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.734944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92d818-fc6b-41f0-ab5f-4ef84136e16b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8813709134a74a7898f5b4cc8fda4544, user_id=563d043ab1ed442b98931bb3c83a5c62, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:36.737711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92d818-fc6b-41f0-ab5f-4ef84136e16b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:36.754902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 810/1627] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:36 2022] PUT /identity/v3/projects/8813709134a74a7898f5b4cc8fda4544/users/563d043ab1ed442b98931bb3c83a5c62/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:36.754902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100c8dd-8042-488b-827d-e43e47730ee2 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.754902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100c8dd-8042-488b-827d-e43e47730ee2 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.754902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100c8dd-8042-488b-827d-e43e47730ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.779200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-9100c8dd-8042-488b-827d-e43e47730ee2 None None] MFA Rules not processed for user `316ef43c6c304bd3a27e085bf4da8a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:36.780564 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-9100c8dd-8042-488b-827d-e43e47730ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:36.811571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7c6bb-00cc-49de-a17f-482af01b8bfe admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:36.858545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 811/1628] 10.222.0.35 () {58 vars in 1058 bytes} [Fri May 13 21:51:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:36.859307 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100c8dd-8042-488b-827d-e43e47730ee2 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.886837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab85fb-0e74-460e-87f2-2f52a733009d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:36.887496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab85fb-0e74-460e-87f2-2f52a733009d service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.888390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab85fb-0e74-460e-87f2-2f52a733009d service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.888704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab85fb-0e74-460e-87f2-2f52a733009d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.925544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100c8dd-8042-488b-827d-e43e47730ee2 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:36.930975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47206c1-fb1b-4671-a754-c1b98b110dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:36.931865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47206c1-fb1b-4671-a754-c1b98b110dff admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:36.932145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47206c1-fb1b-4671-a754-c1b98b110dff admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:36.932418 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47206c1-fb1b-4671-a754-c1b98b110dff admin admin] PATH_INFO: `/v3/projects/f14a4240394542feb3e5391dd674f64c/users/ab066c29a1044cb09bb850dcc59e2a6e/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:36.934405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47206c1-fb1b-4671-a754-c1b98b110dff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14a4240394542feb3e5391dd674f64c, user_id=ab066c29a1044cb09bb850dcc59e2a6e, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:36.937830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47206c1-fb1b-4671-a754-c1b98b110dff admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:36.947428 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab85fb-0e74-460e-87f2-2f52a733009d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:36.949036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab85fb-0e74-460e-87f2-2f52a733009d service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:36.959932 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 812/1629] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:36 2022] PUT /identity/v3/projects/f14a4240394542feb3e5391dd674f64c/users/ab066c29a1044cb09bb850dcc59e2a6e/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:36.959932 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47206c1-fb1b-4671-a754-c1b98b110dff admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.960775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 818/1630] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:36.961873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab85fb-0e74-460e-87f2-2f52a733009d service nova] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:36.974082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab85fb-0e74-460e-87f2-2f52a733009d service nova] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:37.044312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5981c55-e2f0-4705-ab3a-684b67942b57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:37.045137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5981c55-e2f0-4705-ab3a-684b67942b57 service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:37.045525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5981c55-e2f0-4705-ab3a-684b67942b57 service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:37.045836 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47206c1-fb1b-4671-a754-c1b98b110dff admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:37.046087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5981c55-e2f0-4705-ab3a-684b67942b57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:37.052558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6530f-c6f1-4dcf-86b4-04fc25937d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:37.053630 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6530f-c6f1-4dcf-86b4-04fc25937d45 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:37.054195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6530f-c6f1-4dcf-86b4-04fc25937d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:37.054901 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6530f-c6f1-4dcf-86b4-04fc25937d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:37.056054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6530f-c6f1-4dcf-86b4-04fc25937d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:37.057927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6530f-c6f1-4dcf-86b4-04fc25937d45 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:37.078413 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 813/1631] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:37.080875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cffcdc-c614-4e54-85ac-af8f4467cd8b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:37.081309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cffcdc-c614-4e54-85ac-af8f4467cd8b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:37.081609 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cffcdc-c614-4e54-85ac-af8f4467cd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:37.112557 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5981c55-e2f0-4705-ab3a-684b67942b57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:37.114538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5981c55-e2f0-4705-ab3a-684b67942b57 service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:37.120676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-e0cffcdc-c614-4e54-85ac-af8f4467cd8b None None] MFA Rules not processed for user `ab066c29a1044cb09bb850dcc59e2a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:37.123473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-e0cffcdc-c614-4e54-85ac-af8f4467cd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:37.126499 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 819/1632] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:37.127387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5981c55-e2f0-4705-ab3a-684b67942b57 service nova] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:37.141817 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5981c55-e2f0-4705-ab3a-684b67942b57 service nova] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:37.154351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008381a-98bf-4ae7-a185-ea6f20a4f5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:37.155391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008381a-98bf-4ae7-a185-ea6f20a4f5b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:37.155779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008381a-98bf-4ae7-a185-ea6f20a4f5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:37.156172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008381a-98bf-4ae7-a185-ea6f20a4f5b6 admin admin] PATH_INFO: `/v3/projects/8813709134a74a7898f5b4cc8fda4544/users/563d043ab1ed442b98931bb3c83a5c62/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:37.158422 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008381a-98bf-4ae7-a185-ea6f20a4f5b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8813709134a74a7898f5b4cc8fda4544, user_id=563d043ab1ed442b98931bb3c83a5c62, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:37.161399 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008381a-98bf-4ae7-a185-ea6f20a4f5b6 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:37.173031 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 820/1633] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:37 2022] PUT /identity/v3/projects/8813709134a74a7898f5b4cc8fda4544/users/563d043ab1ed442b98931bb3c83a5c62/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:37.184394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e4fd2-4a21-400c-b07d-71eec2452081 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:37.185440 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e4fd2-4a21-400c-b07d-71eec2452081 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:37.185858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e4fd2-4a21-400c-b07d-71eec2452081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:37.228135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 814/1634] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:37.242632 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cffcdc-c614-4e54-85ac-af8f4467cd8b None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:37.246180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-af1e4fd2-4a21-400c-b07d-71eec2452081 None None] MFA Rules not processed for user `563d043ab1ed442b98931bb3c83a5c62`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:37.248821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-af1e4fd2-4a21-400c-b07d-71eec2452081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:37.359346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cffcdc-c614-4e54-85ac-af8f4467cd8b None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:37.407181 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 821/1635] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:37 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:37.409092 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba950d91-ef8f-4077-b6ce-c5fb76d801a6 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:37.409793 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba950d91-ef8f-4077-b6ce-c5fb76d801a6 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:37.420976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba950d91-ef8f-4077-b6ce-c5fb76d801a6 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:37.423316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 822/1636] 10.222.0.35 () {54 vars in 912 bytes} [Fri May 13 21:51:37 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:51:37.424376 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba950d91-ef8f-4077-b6ce-c5fb76d801a6 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:37.442770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fea6e-b07e-44b8-b844-f0b1d1dc81ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:37.443412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fea6e-b07e-44b8-b844-f0b1d1dc81ca service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:37.443565 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fea6e-b07e-44b8-b844-f0b1d1dc81ca service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:37.443926 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fea6e-b07e-44b8-b844-f0b1d1dc81ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:37.545196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba950d91-ef8f-4077-b6ce-c5fb76d801a6 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:37.548403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8fea6e-b07e-44b8-b844-f0b1d1dc81ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:37.550254 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8fea6e-b07e-44b8-b844-f0b1d1dc81ca service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:37.555645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d174d-02e6-4b5b-82cb-96b05394f6b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:37.556322 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d174d-02e6-4b5b-82cb-96b05394f6b1 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:37.556618 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d174d-02e6-4b5b-82cb-96b05394f6b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:37.557064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d174d-02e6-4b5b-82cb-96b05394f6b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:37.565293 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 815/1637] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:37 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:37.566852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd02daf-716a-4519-bbff-bef0d8005dfb None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:37.567142 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd02daf-716a-4519-bbff-bef0d8005dfb None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:37.567428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd02daf-716a-4519-bbff-bef0d8005dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:37.584357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d174d-02e6-4b5b-82cb-96b05394f6b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:37.585826 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d174d-02e6-4b5b-82cb-96b05394f6b1 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:37.589546 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-2bd02daf-716a-4519-bbff-bef0d8005dfb None None] MFA Rules not processed for user `316ef43c6c304bd3a27e085bf4da8a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:37.592124 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2bd02daf-716a-4519-bbff-bef0d8005dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:37.606007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 823/1638] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:37 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:37.607911 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd6634-92cf-48ff-a1f4-7629983f95ea None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:37.608208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd6634-92cf-48ff-a1f4-7629983f95ea None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:37.608806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd6634-92cf-48ff-a1f4-7629983f95ea None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:37.611217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 824/1639] 10.222.0.35 () {54 vars in 912 bytes} [Fri May 13 21:51:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:51:37.624815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85199c99-b027-4985-bb40-872dd37a5a0d None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:37.625484 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85199c99-b027-4985-bb40-872dd37a5a0d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:37.626098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85199c99-b027-4985-bb40-872dd37a5a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:37.679431 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-85199c99-b027-4985-bb40-872dd37a5a0d None None] MFA Rules not processed for user `316ef43c6c304bd3a27e085bf4da8a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:37.679686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 816/1640] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:51:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:37.685638 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-85199c99-b027-4985-bb40-872dd37a5a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:37.712903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 825/1641] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:51:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:51:37.736940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd02daf-716a-4519-bbff-bef0d8005dfb None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:37.757533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85199c99-b027-4985-bb40-872dd37a5a0d None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:37.762324 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd02daf-716a-4519-bbff-bef0d8005dfb None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:37.769799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85199c99-b027-4985-bb40-872dd37a5a0d None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:37.776316 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719306ca-f6f5-4702-80e6-b216abb2f47f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:37.779170 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719306ca-f6f5-4702-80e6-b216abb2f47f service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:37.781592 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86938a-7ed4-4967-a8b7-053ce21ced86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:37.782386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86938a-7ed4-4967-a8b7-053ce21ced86 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:37.782527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719306ca-f6f5-4702-80e6-b216abb2f47f service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:37.782677 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86938a-7ed4-4967-a8b7-053ce21ced86 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:37.782987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86938a-7ed4-4967-a8b7-053ce21ced86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:37.783099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719306ca-f6f5-4702-80e6-b216abb2f47f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:37.808093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86938a-7ed4-4967-a8b7-053ce21ced86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:37.809613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86938a-7ed4-4967-a8b7-053ce21ced86 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:37.836122 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719306ca-f6f5-4702-80e6-b216abb2f47f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:37.838353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719306ca-f6f5-4702-80e6-b216abb2f47f service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:37.838895 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 826/1642] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:51:37.859688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320abc6-f695-47f6-a8bc-117cfd448e17 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:37.859987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320abc6-f695-47f6-a8bc-117cfd448e17 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:37.860270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320abc6-f695-47f6-a8bc-117cfd448e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:37.870445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 817/1643] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:37.912758 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-5320abc6-f695-47f6-a8bc-117cfd448e17 None None] MFA Rules not processed for user `1315f98467d8426094229e3a31c618f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:37.914917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-5320abc6-f695-47f6-a8bc-117cfd448e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:38.018174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 827/1644] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:38.034674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c97e32-dfeb-4440-8351-9d6f5e51e0b9 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:38.036423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c97e32-dfeb-4440-8351-9d6f5e51e0b9 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:38.037016 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c97e32-dfeb-4440-8351-9d6f5e51e0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:38.064325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-23c97e32-dfeb-4440-8351-9d6f5e51e0b9 None None] MFA Rules not processed for user `1315f98467d8426094229e3a31c618f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:38.066695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-23c97e32-dfeb-4440-8351-9d6f5e51e0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:38.079149 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 818/1645] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:38.930675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4e67-8d8c-4c7f-948f-b13690a9a97f None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:38.931202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4e67-8d8c-4c7f-948f-b13690a9a97f None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:38.931295 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4e67-8d8c-4c7f-948f-b13690a9a97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:38.982862 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-bbaf4e67-8d8c-4c7f-948f-b13690a9a97f None None] MFA Rules not processed for user `271827a6cf9344318a279d4daf6e4a09`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:38.984129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-bbaf4e67-8d8c-4c7f-948f-b13690a9a97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:39.061046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c97e32-dfeb-4440-8351-9d6f5e51e0b9 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:39.096029 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c97e32-dfeb-4440-8351-9d6f5e51e0b9 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:39.101911 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 828/1646] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:39.103578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf4e67-8d8c-4c7f-948f-b13690a9a97f None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:39.117574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312725e-f5ff-4f98-977f-db4c9904b3a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:39.118476 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312725e-f5ff-4f98-977f-db4c9904b3a3 service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:39.118789 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312725e-f5ff-4f98-977f-db4c9904b3a3 service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:39.119078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312725e-f5ff-4f98-977f-db4c9904b3a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:39.159554 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312725e-f5ff-4f98-977f-db4c9904b3a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:39.165393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312725e-f5ff-4f98-977f-db4c9904b3a3 service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:39.225677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 819/1647] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:39.243609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf4e67-8d8c-4c7f-948f-b13690a9a97f None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:39.254733 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bb13c-a089-4462-901a-a0cc4e598d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:39.260161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bb13c-a089-4462-901a-a0cc4e598d22 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:39.260460 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bb13c-a089-4462-901a-a0cc4e598d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:39.260744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bb13c-a089-4462-901a-a0cc4e598d22 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:39.261478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2bb13c-a089-4462-901a-a0cc4e598d22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:39.263741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2bb13c-a089-4462-901a-a0cc4e598d22 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:39.322573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 829/1648] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:39 2022] POST /identity/v3/projects => generated 403 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:51:39.332208 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312725e-f5ff-4f98-977f-db4c9904b3a3 service nova] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:39.377984 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312725e-f5ff-4f98-977f-db4c9904b3a3 service nova] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:39.488592 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bb13c-a089-4462-901a-a0cc4e598d22 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:39.520788 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3c2f2-8c9a-48c7-8f49-f681202cb2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:39.521475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3c2f2-8c9a-48c7-8f49-f681202cb2a5 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:39.521760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3c2f2-8c9a-48c7-8f49-f681202cb2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:39.522141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3c2f2-8c9a-48c7-8f49-f681202cb2a5 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:39.522880 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3c2f2-8c9a-48c7-8f49-f681202cb2a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:39.524812 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3c2f2-8c9a-48c7-8f49-f681202cb2a5 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:39.574630 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 820/1649] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:39 2022] POST /identity/v3/users => generated 308 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:39.576945 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3c2f2-8c9a-48c7-8f49-f681202cb2a5 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:39.590312 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3c2f2-8c9a-48c7-8f49-f681202cb2a5 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:39.600290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17eea2-20c2-4d1f-b126-acf6c082474b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:39.601005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17eea2-20c2-4d1f-b126-acf6c082474b admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:39.601221 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17eea2-20c2-4d1f-b126-acf6c082474b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:39.601506 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17eea2-20c2-4d1f-b126-acf6c082474b admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:39.605350 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17eea2-20c2-4d1f-b126-acf6c082474b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:39.607201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17eea2-20c2-4d1f-b126-acf6c082474b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:39.630125 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bb13c-a089-4462-901a-a0cc4e598d22 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:39.643351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 821/1650] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:39.646620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb00d4c-9a5c-4ecc-b413-0c00bec1ec8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:39.646620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb00d4c-9a5c-4ecc-b413-0c00bec1ec8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:39.646620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb00d4c-9a5c-4ecc-b413-0c00bec1ec8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:39.646620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb00d4c-9a5c-4ecc-b413-0c00bec1ec8c admin admin] PATH_INFO: `/v3/users/a089cef485964a0b9483391ba5ec3037` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:39.648676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17eea2-20c2-4d1f-b126-acf6c082474b admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:39.655651 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb00d4c-9a5c-4ecc-b413-0c00bec1ec8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a089cef485964a0b9483391ba5ec3037)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:39.656763 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: WARNING py.warnings [None req-2bb00d4c-9a5c-4ecc-b413-0c00bec1ec8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 21:51:39.656763 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: warnings.warn(msg) May 13 21:51:39.656763 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: May 13 21:51:39.658018 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb00d4c-9a5c-4ecc-b413-0c00bec1ec8c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:39.673817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17eea2-20c2-4d1f-b126-acf6c082474b admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:39.682513 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77241035-e1cc-4ec0-b57f-be88b8c09037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:39.683900 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77241035-e1cc-4ec0-b57f-be88b8c09037 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:39.684591 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77241035-e1cc-4ec0-b57f-be88b8c09037 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:39.685041 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77241035-e1cc-4ec0-b57f-be88b8c09037 admin admin] PATH_INFO: `/v3/projects/14b40140d4ca4ee4b199e776dcfb652e/users/8e28e4fb3479472c9f831354bc529ccc/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:39.750218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77241035-e1cc-4ec0-b57f-be88b8c09037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b40140d4ca4ee4b199e776dcfb652e, user_id=8e28e4fb3479472c9f831354bc529ccc, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:39.756174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77241035-e1cc-4ec0-b57f-be88b8c09037 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:39.781156 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-2bb00d4c-9a5c-4ecc-b413-0c00bec1ec8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a089cef485964a0b9483391ba5ec3037'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:39.792471 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 822/1651] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:39 2022] PUT /identity/v3/projects/14b40140d4ca4ee4b199e776dcfb652e/users/8e28e4fb3479472c9f831354bc529ccc/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:39.806432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77241035-e1cc-4ec0-b57f-be88b8c09037 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:39.843487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-2bb00d4c-9a5c-4ecc-b413-0c00bec1ec8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a089cef485964a0b9483391ba5ec3037'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:39.846501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-2bb00d4c-9a5c-4ecc-b413-0c00bec1ec8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a089cef485964a0b9483391ba5ec3037'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:39.848148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-2bb00d4c-9a5c-4ecc-b413-0c00bec1ec8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a089cef485964a0b9483391ba5ec3037'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:39.868594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-2bb00d4c-9a5c-4ecc-b413-0c00bec1ec8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a089cef485964a0b9483391ba5ec3037'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:39.898457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 830/1652] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:51:39 2022] DELETE /identity/v3/users/a089cef485964a0b9483391ba5ec3037 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:39.903140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab180d-6d35-4595-a0b3-6488f39fefd6 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:39.903648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab180d-6d35-4595-a0b3-6488f39fefd6 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:39.904117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab180d-6d35-4595-a0b3-6488f39fefd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:39.924501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77241035-e1cc-4ec0-b57f-be88b8c09037 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:39.952571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d4ab180d-6d35-4595-a0b3-6488f39fefd6 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:39.956179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d4ab180d-6d35-4595-a0b3-6488f39fefd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:40.027254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 831/1653] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:51:40.028683 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8daac-d32a-4252-8f5b-0313be44e284 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:40.029175 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8daac-d32a-4252-8f5b-0313be44e284 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:40.029633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8daac-d32a-4252-8f5b-0313be44e284 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:40.042861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 832/1654] 10.222.0.35 () {54 vars in 912 bytes} [Fri May 13 21:51:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:51:40.044278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8daac-d32a-4252-8f5b-0313be44e284 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:40.046761 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e4d8daac-d32a-4252-8f5b-0313be44e284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:40.065017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8daac-d32a-4252-8f5b-0313be44e284 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:40.070530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0810fb-589b-404a-8b39-af5f1c1bbd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:40.071745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0810fb-589b-404a-8b39-af5f1c1bbd92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:40.072219 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0810fb-589b-404a-8b39-af5f1c1bbd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:40.072832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0810fb-589b-404a-8b39-af5f1c1bbd92 admin admin] PATH_INFO: `/v3/users/aa7cac0c3ed34d46ba359f89ee8f68c1` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:40.074621 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0810fb-589b-404a-8b39-af5f1c1bbd92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa7cac0c3ed34d46ba359f89ee8f68c1)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:40.078061 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0810fb-589b-404a-8b39-af5f1c1bbd92 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:40.159904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ca0810fb-589b-404a-8b39-af5f1c1bbd92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa7cac0c3ed34d46ba359f89ee8f68c1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:40.177209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ca0810fb-589b-404a-8b39-af5f1c1bbd92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa7cac0c3ed34d46ba359f89ee8f68c1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:40.178162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ca0810fb-589b-404a-8b39-af5f1c1bbd92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa7cac0c3ed34d46ba359f89ee8f68c1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:40.181283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ca0810fb-589b-404a-8b39-af5f1c1bbd92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa7cac0c3ed34d46ba359f89ee8f68c1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:40.192860 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ca0810fb-589b-404a-8b39-af5f1c1bbd92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa7cac0c3ed34d46ba359f89ee8f68c1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:40.207667 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e388b28-f78b-4beb-8c11-92a201f1023e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:40.209352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e388b28-f78b-4beb-8c11-92a201f1023e admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:40.210482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e388b28-f78b-4beb-8c11-92a201f1023e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:40.211110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e388b28-f78b-4beb-8c11-92a201f1023e admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:40.212124 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e388b28-f78b-4beb-8c11-92a201f1023e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:40.214082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 833/1655] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:51:40 2022] DELETE /identity/v3/users/aa7cac0c3ed34d46ba359f89ee8f68c1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:40.215262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0810fb-589b-404a-8b39-af5f1c1bbd92 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:40.217135 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ca0810fb-589b-404a-8b39-af5f1c1bbd92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:40.218124 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e388b28-f78b-4beb-8c11-92a201f1023e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:40.241174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 823/1656] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:40.243351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8583c-deb0-4446-8fbb-b7a839650f82 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:40.244619 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8583c-deb0-4446-8fbb-b7a839650f82 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:40.245135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8583c-deb0-4446-8fbb-b7a839650f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:40.310074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-76c8583c-deb0-4446-8fbb-b7a839650f82 None None] MFA Rules not processed for user `c4df058606c448a1bb757020e869ac33`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:40.311916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-76c8583c-deb0-4446-8fbb-b7a839650f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:40.340933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0810fb-589b-404a-8b39-af5f1c1bbd92 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:40.346587 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc0985-69ed-422d-a072-42716b6ddcd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:40.347651 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc0985-69ed-422d-a072-42716b6ddcd4 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:40.348187 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc0985-69ed-422d-a072-42716b6ddcd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:40.348734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc0985-69ed-422d-a072-42716b6ddcd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:40.351029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-40dc0985-69ed-422d-a072-42716b6ddcd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:40.374741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 824/1657] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:51:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:40.382297 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc3a5b-07a0-4e2d-aa9f-ca5fec4fe797 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:40.382645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc3a5b-07a0-4e2d-aa9f-ca5fec4fe797 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:40.382905 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc3a5b-07a0-4e2d-aa9f-ca5fec4fe797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:40.405095 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-d8bc3a5b-07a0-4e2d-aa9f-ca5fec4fe797 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:40.406735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d8bc3a5b-07a0-4e2d-aa9f-ca5fec4fe797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:40.507327 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 825/1658] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:40.508177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc3a5b-07a0-4e2d-aa9f-ca5fec4fe797 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:40.514456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d8bc3a5b-07a0-4e2d-aa9f-ca5fec4fe797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:40.525089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc0985-69ed-422d-a072-42716b6ddcd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:40.527327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc0985-69ed-422d-a072-42716b6ddcd4 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:40.538491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc3a5b-07a0-4e2d-aa9f-ca5fec4fe797 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:40.547703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 834/1659] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:40.548303 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc0985-69ed-422d-a072-42716b6ddcd4 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:40.555051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e49a27-cec8-4a72-adf8-d629d790dd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:40.555963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e49a27-cec8-4a72-adf8-d629d790dd80 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:40.556253 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e49a27-cec8-4a72-adf8-d629d790dd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:40.556531 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e49a27-cec8-4a72-adf8-d629d790dd80 admin admin] PATH_INFO: `/v3/projects/14b40140d4ca4ee4b199e776dcfb652e/users/8e28e4fb3479472c9f831354bc529ccc/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:40.560939 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e49a27-cec8-4a72-adf8-d629d790dd80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b40140d4ca4ee4b199e776dcfb652e, user_id=8e28e4fb3479472c9f831354bc529ccc, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:40.563673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e49a27-cec8-4a72-adf8-d629d790dd80 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:40.569838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc0985-69ed-422d-a072-42716b6ddcd4 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:40.578882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b69cc-7034-4ada-87dc-ca2857ca5cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:40.579737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b69cc-7034-4ada-87dc-ca2857ca5cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:40.579859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b69cc-7034-4ada-87dc-ca2857ca5cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:40.580095 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b69cc-7034-4ada-87dc-ca2857ca5cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:40.582915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 826/1660] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:40 2022] PUT /identity/v3/projects/14b40140d4ca4ee4b199e776dcfb652e/users/8e28e4fb3479472c9f831354bc529ccc/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:40.584110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e49a27-cec8-4a72-adf8-d629d790dd80 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:40.599370 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b69cc-7034-4ada-87dc-ca2857ca5cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:40.601300 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b69cc-7034-4ada-87dc-ca2857ca5cf0 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:40.614880 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 835/1661] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:40.690863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e49a27-cec8-4a72-adf8-d629d790dd80 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:40.695399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3660b3a-83aa-4884-b821-07303e81949a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:40.696451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3660b3a-83aa-4884-b821-07303e81949a service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:40.696749 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3660b3a-83aa-4884-b821-07303e81949a service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:40.697027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3660b3a-83aa-4884-b821-07303e81949a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:40.948635 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3660b3a-83aa-4884-b821-07303e81949a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:40.957381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3660b3a-83aa-4884-b821-07303e81949a service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:40.976107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 827/1662] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:41.713248 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e901a8-93ff-4773-95d3-58775fdfa1ee None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:41.714226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e901a8-93ff-4773-95d3-58775fdfa1ee None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:41.715800 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e901a8-93ff-4773-95d3-58775fdfa1ee None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:41.723136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc77dd-b2c0-448c-ad67-5072f0f33881 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:41.723668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc77dd-b2c0-448c-ad67-5072f0f33881 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:41.723754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc77dd-b2c0-448c-ad67-5072f0f33881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:41.724492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 836/1663] 10.222.0.35 () {54 vars in 960 bytes} [Fri May 13 21:51:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:51:41.750980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-f2dc77dd-b2c0-448c-ad67-5072f0f33881 None None] MFA Rules not processed for user `8124ab8abcc640c5a48b3e0216afa48d`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:41.752237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f2dc77dd-b2c0-448c-ad67-5072f0f33881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:41.832218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 828/1664] 10.222.0.35 () {58 vars in 1057 bytes} [Fri May 13 21:51:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:51:41.839324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e901a8-93ff-4773-95d3-58775fdfa1ee None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:41.864176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e901a8-93ff-4773-95d3-58775fdfa1ee None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:41.873641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437c488-0062-4f08-b2f8-0fbaa4321559 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8124ab8abcc640c5a48b3e0216afa48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:41.874677 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437c488-0062-4f08-b2f8-0fbaa4321559 service placement] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:41.875222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437c488-0062-4f08-b2f8-0fbaa4321559 service placement] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:41.875719 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437c488-0062-4f08-b2f8-0fbaa4321559 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:41.878741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7437c488-0062-4f08-b2f8-0fbaa4321559 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:41.897512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437c488-0062-4f08-b2f8-0fbaa4321559 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:41.899176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437c488-0062-4f08-b2f8-0fbaa4321559 service placement] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:41.929703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 837/1665] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:42.120454 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc77dd-b2c0-448c-ad67-5072f0f33881 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:42.121707 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f2dc77dd-b2c0-448c-ad67-5072f0f33881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:42.146780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc77dd-b2c0-448c-ad67-5072f0f33881 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:42.165304 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:42.166312 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:42.166485 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:42.166784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] PATH_INFO: `/v3/projects/26f61a0463f04099abea79ab789f2fc9` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:42.171233 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26f61a0463f04099abea79ab789f2fc9)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:42.172349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: WARNING py.warnings [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 21:51:42.172349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: warnings.warn(msg) May 13 21:51:42.172349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: May 13 21:51:42.173864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:42.180520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437c488-0062-4f08-b2f8-0fbaa4321559 service placement] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:42.182116 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7437c488-0062-4f08-b2f8-0fbaa4321559 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:42.216678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437c488-0062-4f08-b2f8-0fbaa4321559 service placement] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:42.240345 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:42.241235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:42.241784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:42.242061 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] PATH_INFO: `/v3/projects/79bcd2be37fc4969b1af6424024343aa` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:42.248437 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] RBAC: Authorizing `identity:delete_project(project_id=79bcd2be37fc4969b1af6424024343aa)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:42.250409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:42.318646 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26f61a0463f04099abea79ab789f2fc9', 'request_id': 'req-53903209-986d-4e07-ba0a-b9d45598fad5'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.352565 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26f61a0463f04099abea79ab789f2fc9', 'request_id': 'req-53903209-986d-4e07-ba0a-b9d45598fad5'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.365059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26f61a0463f04099abea79ab789f2fc9', 'request_id': 'req-53903209-986d-4e07-ba0a-b9d45598fad5'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.415292 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] The token cache is being invalidate because project 26f61a0463f04099abea79ab789f2fc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:51:42.415292 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.416054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.418448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 829/1666] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:51:42 2022] DELETE /identity/v3/projects/26f61a0463f04099abea79ab789f2fc9 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:42.423059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:42.424791 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:42.448453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79bcd2be37fc4969b1af6424024343aa', 'request_id': 'req-7b1b06ec-a29a-423d-a656-165cf047b86d'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.458318 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79bcd2be37fc4969b1af6424024343aa', 'request_id': 'req-7b1b06ec-a29a-423d-a656-165cf047b86d'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.479491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79bcd2be37fc4969b1af6424024343aa', 'request_id': 'req-7b1b06ec-a29a-423d-a656-165cf047b86d'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.508811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] The token cache is being invalidate because project 79bcd2be37fc4969b1af6424024343aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:51:42.509360 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.511080 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.514665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 838/1667] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:51:42 2022] DELETE /identity/v3/projects/79bcd2be37fc4969b1af6424024343aa => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:51:42.519475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:42.522793 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:42.574039 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53903209-986d-4e07-ba0a-b9d45598fad5 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:42.732029 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df42f9-4973-4a54-a523-7f680055c4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:42.733865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df42f9-4973-4a54-a523-7f680055c4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:42.734550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df42f9-4973-4a54-a523-7f680055c4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:42.735362 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df42f9-4973-4a54-a523-7f680055c4ab admin admin] PATH_INFO: `/v3/users/563d043ab1ed442b98931bb3c83a5c62` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:42.736976 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df42f9-4973-4a54-a523-7f680055c4ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=563d043ab1ed442b98931bb3c83a5c62)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:42.739202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df42f9-4973-4a54-a523-7f680055c4ab admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:42.779282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b06ec-a29a-423d-a656-165cf047b86d admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:42.795954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb7394-b8f6-47d2-80c8-31145eb16c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:42.796906 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb7394-b8f6-47d2-80c8-31145eb16c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:42.797212 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb7394-b8f6-47d2-80c8-31145eb16c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:42.797486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb7394-b8f6-47d2-80c8-31145eb16c57 admin admin] PATH_INFO: `/v3/users/ab066c29a1044cb09bb850dcc59e2a6e` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:42.803056 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb7394-b8f6-47d2-80c8-31145eb16c57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab066c29a1044cb09bb850dcc59e2a6e)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:42.807608 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb7394-b8f6-47d2-80c8-31145eb16c57 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:42.831146 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-41df42f9-4973-4a54-a523-7f680055c4ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563d043ab1ed442b98931bb3c83a5c62'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.884543 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-41df42f9-4973-4a54-a523-7f680055c4ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563d043ab1ed442b98931bb3c83a5c62'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.888217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-41df42f9-4973-4a54-a523-7f680055c4ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563d043ab1ed442b98931bb3c83a5c62'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.890719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-41df42f9-4973-4a54-a523-7f680055c4ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563d043ab1ed442b98931bb3c83a5c62'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.916479 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-41df42f9-4973-4a54-a523-7f680055c4ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563d043ab1ed442b98931bb3c83a5c62'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.963607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 830/1668] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:51:42 2022] DELETE /identity/v3/users/563d043ab1ed442b98931bb3c83a5c62 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:42.972476 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-4ddb7394-b8f6-47d2-80c8-31145eb16c57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab066c29a1044cb09bb850dcc59e2a6e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.992792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-4ddb7394-b8f6-47d2-80c8-31145eb16c57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab066c29a1044cb09bb850dcc59e2a6e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.993874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-4ddb7394-b8f6-47d2-80c8-31145eb16c57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab066c29a1044cb09bb850dcc59e2a6e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:42.994843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-4ddb7394-b8f6-47d2-80c8-31145eb16c57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab066c29a1044cb09bb850dcc59e2a6e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:43.008159 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-4ddb7394-b8f6-47d2-80c8-31145eb16c57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab066c29a1044cb09bb850dcc59e2a6e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:43.015892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240fb2c-f86d-438d-ac78-bab254a532c8 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:43.016579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240fb2c-f86d-438d-ac78-bab254a532c8 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:43.017025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240fb2c-f86d-438d-ac78-bab254a532c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:43.045105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 839/1669] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:51:42 2022] DELETE /identity/v3/users/ab066c29a1044cb09bb850dcc59e2a6e => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:43.061327 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-8240fb2c-f86d-438d-ac78-bab254a532c8 None None] MFA Rules not processed for user `8e28e4fb3479472c9f831354bc529ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:43.063119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8240fb2c-f86d-438d-ac78-bab254a532c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:43.251399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 831/1670] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:43 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:43.930724 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb7394-b8f6-47d2-80c8-31145eb16c57 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:43.932318 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-4ddb7394-b8f6-47d2-80c8-31145eb16c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:44.058606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb7394-b8f6-47d2-80c8-31145eb16c57 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:44.065484 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee8aa8-6eb4-49aa-846a-c42842a20f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:44.066593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee8aa8-6eb4-49aa-846a-c42842a20f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:44.067023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee8aa8-6eb4-49aa-846a-c42842a20f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:44.067367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee8aa8-6eb4-49aa-846a-c42842a20f19 admin admin] PATH_INFO: `/v3/projects/8813709134a74a7898f5b4cc8fda4544` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:44.068780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee8aa8-6eb4-49aa-846a-c42842a20f19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8813709134a74a7898f5b4cc8fda4544)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:44.070992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee8aa8-6eb4-49aa-846a-c42842a20f19 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:44.158170 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6eee8aa8-6eb4-49aa-846a-c42842a20f19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8813709134a74a7898f5b4cc8fda4544', 'request_id': 'req-6eee8aa8-6eb4-49aa-846a-c42842a20f19'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:44.166037 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6eee8aa8-6eb4-49aa-846a-c42842a20f19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8813709134a74a7898f5b4cc8fda4544', 'request_id': 'req-6eee8aa8-6eb4-49aa-846a-c42842a20f19'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:44.178461 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6eee8aa8-6eb4-49aa-846a-c42842a20f19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8813709134a74a7898f5b4cc8fda4544', 'request_id': 'req-6eee8aa8-6eb4-49aa-846a-c42842a20f19'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:44.201686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6eee8aa8-6eb4-49aa-846a-c42842a20f19 admin admin] The token cache is being invalidate because project 8813709134a74a7898f5b4cc8fda4544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:51:44.202198 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6eee8aa8-6eb4-49aa-846a-c42842a20f19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:44.203161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6eee8aa8-6eb4-49aa-846a-c42842a20f19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:44.205379 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 840/1671] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:51:43 2022] DELETE /identity/v3/projects/8813709134a74a7898f5b4cc8fda4544 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:44.210168 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50963a4-12d8-4e12-998e-8ca8a014df1a None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:44.210595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50963a4-12d8-4e12-998e-8ca8a014df1a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:44.210840 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50963a4-12d8-4e12-998e-8ca8a014df1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:44.248587 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-d50963a4-12d8-4e12-998e-8ca8a014df1a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:44.249912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d50963a4-12d8-4e12-998e-8ca8a014df1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:44.381563 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 832/1672] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:44.389613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e50c94-e26d-4917-a284-89d6bcd96756 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:44.389899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e50c94-e26d-4917-a284-89d6bcd96756 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:44.390175 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e50c94-e26d-4917-a284-89d6bcd96756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:44.418852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-21e50c94-e26d-4917-a284-89d6bcd96756 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:44.419730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-21e50c94-e26d-4917-a284-89d6bcd96756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:44.432112 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 841/1673] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:51:44.433312 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50963a4-12d8-4e12-998e-8ca8a014df1a None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:44.453801 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50963a4-12d8-4e12-998e-8ca8a014df1a None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:44.459562 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a953f3d-62b3-47bf-9218-dc6bfcea86a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:44.460511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a953f3d-62b3-47bf-9218-dc6bfcea86a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:44.461130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a953f3d-62b3-47bf-9218-dc6bfcea86a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:44.462181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a953f3d-62b3-47bf-9218-dc6bfcea86a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:44.463975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a953f3d-62b3-47bf-9218-dc6bfcea86a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:51:44.465101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a953f3d-62b3-47bf-9218-dc6bfcea86a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:44.467183 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a953f3d-62b3-47bf-9218-dc6bfcea86a9 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:44.488017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 833/1674] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:51:44.492636 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e50c94-e26d-4917-a284-89d6bcd96756 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:44.514601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e50c94-e26d-4917-a284-89d6bcd96756 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:44.531156 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93045df-2c06-4b1c-a6ab-db46d8b418f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:44.537220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93045df-2c06-4b1c-a6ab-db46d8b418f2 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:44.538413 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93045df-2c06-4b1c-a6ab-db46d8b418f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:44.538869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93045df-2c06-4b1c-a6ab-db46d8b418f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:44.543484 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93045df-2c06-4b1c-a6ab-db46d8b418f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:44.550841 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93045df-2c06-4b1c-a6ab-db46d8b418f2 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:44.603365 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 842/1675] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:44 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:44.605846 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a953f3d-62b3-47bf-9218-dc6bfcea86a9 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:44.636710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a953f3d-62b3-47bf-9218-dc6bfcea86a9 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:44.750005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f595e9-53cd-4dc4-90e6-724a5ba7c94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:44.753609 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f595e9-53cd-4dc4-90e6-724a5ba7c94e admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:44.753875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f595e9-53cd-4dc4-90e6-724a5ba7c94e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:44.754319 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f595e9-53cd-4dc4-90e6-724a5ba7c94e admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:44.757354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f595e9-53cd-4dc4-90e6-724a5ba7c94e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:44.762511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f595e9-53cd-4dc4-90e6-724a5ba7c94e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:44.799490 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 834/1676] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:44 2022] POST /identity/v3/users => generated 291 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:44.800992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93045df-2c06-4b1c-a6ab-db46d8b418f2 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:44.816106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93045df-2c06-4b1c-a6ab-db46d8b418f2 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:44.824301 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f595e9-53cd-4dc4-90e6-724a5ba7c94e admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:44.825687 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cc4e1-20ee-40c3-b1b4-56ab5faae0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:44.826159 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-91f595e9-53cd-4dc4-90e6-724a5ba7c94e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:44.826698 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cc4e1-20ee-40c3-b1b4-56ab5faae0fb admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:44.827130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cc4e1-20ee-40c3-b1b4-56ab5faae0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:44.827583 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cc4e1-20ee-40c3-b1b4-56ab5faae0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:44.828433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cc4e1-20ee-40c3-b1b4-56ab5faae0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:44.831488 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cc4e1-20ee-40c3-b1b4-56ab5faae0fb admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:44.843368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 843/1677] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:44.848086 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cc4e1-20ee-40c3-b1b4-56ab5faae0fb admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:44.855796 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f595e9-53cd-4dc4-90e6-724a5ba7c94e admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:44.861579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cc4e1-20ee-40c3-b1b4-56ab5faae0fb admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:44.868042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374496f3-9de1-4f60-bfd7-8498134ea084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:44.868830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374496f3-9de1-4f60-bfd7-8498134ea084 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:44.869115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374496f3-9de1-4f60-bfd7-8498134ea084 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:44.869394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374496f3-9de1-4f60-bfd7-8498134ea084 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:44.870210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374496f3-9de1-4f60-bfd7-8498134ea084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:44.871087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:44.872425 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374496f3-9de1-4f60-bfd7-8498134ea084 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:44.874145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:44.874451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:44.874770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3 admin admin] PATH_INFO: `/v3/projects/f14a4240394542feb3e5391dd674f64c` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:44.877591 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f14a4240394542feb3e5391dd674f64c)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:44.881974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:44.890934 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 844/1678] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:51:44.891900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374496f3-9de1-4f60-bfd7-8498134ea084 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:44.913822 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374496f3-9de1-4f60-bfd7-8498134ea084 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:44.920697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e9a19-5bf9-45e5-a823-d8ff24f03409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:44.922357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e9a19-5bf9-45e5-a823-d8ff24f03409 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:44.922357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e9a19-5bf9-45e5-a823-d8ff24f03409 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:44.922523 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e9a19-5bf9-45e5-a823-d8ff24f03409 admin admin] PATH_INFO: `/v3/projects/48cfbf08a65a4814bb083dd255d0dd13/users/26ea13fb1bc54878a47c201e9bde9996/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:44.970977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e9a19-5bf9-45e5-a823-d8ff24f03409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48cfbf08a65a4814bb083dd255d0dd13, user_id=26ea13fb1bc54878a47c201e9bde9996, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:44.987668 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e9a19-5bf9-45e5-a823-d8ff24f03409 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:45.000241 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f14a4240394542feb3e5391dd674f64c', 'request_id': 'req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:45.008197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f14a4240394542feb3e5391dd674f64c', 'request_id': 'req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:45.025540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 845/1679] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:44 2022] PUT /identity/v3/projects/48cfbf08a65a4814bb083dd255d0dd13/users/26ea13fb1bc54878a47c201e9bde9996/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:45.026324 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f14a4240394542feb3e5391dd674f64c', 'request_id': 'req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:45.038520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3 admin admin] The token cache is being invalidate because project f14a4240394542feb3e5391dd674f64c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:51:45.038520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:45.043132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-981ddba6-3a44-4722-beb2-66ad2c9ac7c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:45.046575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 835/1680] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:51:44 2022] DELETE /identity/v3/projects/f14a4240394542feb3e5391dd674f64c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:45.051605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886c7f2-fac0-48b4-944e-116dbdd40697 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:45.052346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886c7f2-fac0-48b4-944e-116dbdd40697 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:45.052649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886c7f2-fac0-48b4-944e-116dbdd40697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:45.053738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd432d-70dd-44a9-90b3-e8136ffb3dfa None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:45.055783 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd432d-70dd-44a9-90b3-e8136ffb3dfa None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:45.056714 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd432d-70dd-44a9-90b3-e8136ffb3dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:45.088477 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-c886c7f2-fac0-48b4-944e-116dbdd40697 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:45.089699 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c886c7f2-fac0-48b4-944e-116dbdd40697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:45.108689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-aadd432d-70dd-44a9-90b3-e8136ffb3dfa None None] MFA Rules not processed for user `26ea13fb1bc54878a47c201e9bde9996`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:45.111011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-aadd432d-70dd-44a9-90b3-e8136ffb3dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:45.203506 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 836/1681] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:51:45.204647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886c7f2-fac0-48b4-944e-116dbdd40697 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:45.206185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c886c7f2-fac0-48b4-944e-116dbdd40697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:45.226072 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886c7f2-fac0-48b4-944e-116dbdd40697 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:45.232826 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106105d8-6208-4100-af4c-1433c7e01bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:45.234051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106105d8-6208-4100-af4c-1433c7e01bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:45.234351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106105d8-6208-4100-af4c-1433c7e01bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:45.234634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106105d8-6208-4100-af4c-1433c7e01bf1 admin admin] PATH_INFO: `/v3/users/ff694cc1ed2d4b99a2dbfaeffcb071d0` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:45.236355 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106105d8-6208-4100-af4c-1433c7e01bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff694cc1ed2d4b99a2dbfaeffcb071d0)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:45.238315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106105d8-6208-4100-af4c-1433c7e01bf1 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:45.281321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 846/1682] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:45.283137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace59f35-f27b-4820-9844-bca2739fbac5 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:45.287110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace59f35-f27b-4820-9844-bca2739fbac5 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:45.287692 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace59f35-f27b-4820-9844-bca2739fbac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:45.315420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-106105d8-6208-4100-af4c-1433c7e01bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff694cc1ed2d4b99a2dbfaeffcb071d0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:45.321502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-ace59f35-f27b-4820-9844-bca2739fbac5 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:45.323329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ace59f35-f27b-4820-9844-bca2739fbac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:45.334233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 847/1683] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:45.336855 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace59f35-f27b-4820-9844-bca2739fbac5 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:45.338469 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-106105d8-6208-4100-af4c-1433c7e01bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff694cc1ed2d4b99a2dbfaeffcb071d0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:45.339480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-106105d8-6208-4100-af4c-1433c7e01bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff694cc1ed2d4b99a2dbfaeffcb071d0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:45.340256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-106105d8-6208-4100-af4c-1433c7e01bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff694cc1ed2d4b99a2dbfaeffcb071d0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:45.351381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ace59f35-f27b-4820-9844-bca2739fbac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:45.353463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-106105d8-6208-4100-af4c-1433c7e01bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff694cc1ed2d4b99a2dbfaeffcb071d0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:45.379323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 837/1684] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:51:45 2022] DELETE /identity/v3/users/ff694cc1ed2d4b99a2dbfaeffcb071d0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:45.379632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace59f35-f27b-4820-9844-bca2739fbac5 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:45.380620 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106105d8-6208-4100-af4c-1433c7e01bf1 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:45.383825 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-106105d8-6208-4100-af4c-1433c7e01bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:45.466173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106105d8-6208-4100-af4c-1433c7e01bf1 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:45.479058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480fb0ff-d813-4778-b156-943b512d42d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:45.481399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fb0ff-d813-4778-b156-943b512d42d5 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:45.481707 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fb0ff-d813-4778-b156-943b512d42d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:45.482133 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fb0ff-d813-4778-b156-943b512d42d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:45.483378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480fb0ff-d813-4778-b156-943b512d42d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:51:45.483378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480fb0ff-d813-4778-b156-943b512d42d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:45.484889 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480fb0ff-d813-4778-b156-943b512d42d5 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:45.498815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fc821-047c-4e76-a77c-d51441058f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:45.500677 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fc821-047c-4e76-a77c-d51441058f20 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:45.501822 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fc821-047c-4e76-a77c-d51441058f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:45.502589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 838/1685] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:51:45.503047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fc821-047c-4e76-a77c-d51441058f20 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:45.504137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fc821-047c-4e76-a77c-d51441058f20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:45.504257 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ac9a-14d1-452a-915d-6ccbebf7527d None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:45.505195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ac9a-14d1-452a-915d-6ccbebf7527d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:45.505485 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ac9a-14d1-452a-915d-6ccbebf7527d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:45.508660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fc821-047c-4e76-a77c-d51441058f20 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:45.537315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-b427ac9a-14d1-452a-915d-6ccbebf7527d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:45.538621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b427ac9a-14d1-452a-915d-6ccbebf7527d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:45.547662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 848/1686] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:45 2022] POST /identity/v3/projects => generated 381 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:45.547662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fc821-047c-4e76-a77c-d51441058f20 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:45.550575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 839/1687] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:45.553681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427ac9a-14d1-452a-915d-6ccbebf7527d None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:45.554164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b427ac9a-14d1-452a-915d-6ccbebf7527d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:45.569773 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427ac9a-14d1-452a-915d-6ccbebf7527d None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:45.722312 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e2f3f-3970-46a2-a1c5-bbdea12ed4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:45.722957 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e2f3f-3970-46a2-a1c5-bbdea12ed4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:45.723236 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e2f3f-3970-46a2-a1c5-bbdea12ed4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:45.723515 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e2f3f-3970-46a2-a1c5-bbdea12ed4b7 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:45.724208 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e2f3f-3970-46a2-a1c5-bbdea12ed4b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:45.726152 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e2f3f-3970-46a2-a1c5-bbdea12ed4b7 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:45.755621 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fc821-047c-4e76-a77c-d51441058f20 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:45.761905 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7581eef-5d30-4acc-a7bf-6b4dbccb4bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:45.763379 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7581eef-5d30-4acc-a7bf-6b4dbccb4bac admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:45.763886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7581eef-5d30-4acc-a7bf-6b4dbccb4bac admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:45.764212 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7581eef-5d30-4acc-a7bf-6b4dbccb4bac admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:45.764954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7581eef-5d30-4acc-a7bf-6b4dbccb4bac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:45.767870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7581eef-5d30-4acc-a7bf-6b4dbccb4bac admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:45.774365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 840/1688] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:45 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:45.774365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e2f3f-3970-46a2-a1c5-bbdea12ed4b7 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:45.776056 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c42e2f3f-3970-46a2-a1c5-bbdea12ed4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:45.839774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 849/1689] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:45 2022] POST /identity/v3/projects => generated 363 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:45.839774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7581eef-5d30-4acc-a7bf-6b4dbccb4bac admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:45.938904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e2f3f-3970-46a2-a1c5-bbdea12ed4b7 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:45.951597 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7581eef-5d30-4acc-a7bf-6b4dbccb4bac admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:45.959854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dcf67-7bf8-4048-9036-6bef6d3113bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:45.960963 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dcf67-7bf8-4048-9036-6bef6d3113bc admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:45.961423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dcf67-7bf8-4048-9036-6bef6d3113bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:45.961889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dcf67-7bf8-4048-9036-6bef6d3113bc admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:45.963030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dcf67-7bf8-4048-9036-6bef6d3113bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:45.965058 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dcf67-7bf8-4048-9036-6bef6d3113bc admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:45.977685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 850/1690] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:45.980646 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dcf67-7bf8-4048-9036-6bef6d3113bc admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:46.003327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dcf67-7bf8-4048-9036-6bef6d3113bc admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:46.016727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13455332-9d6d-401b-9039-2ecb3a13ac80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:46.017784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13455332-9d6d-401b-9039-2ecb3a13ac80 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:46.018358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13455332-9d6d-401b-9039-2ecb3a13ac80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:46.018854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13455332-9d6d-401b-9039-2ecb3a13ac80 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:46.019738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13455332-9d6d-401b-9039-2ecb3a13ac80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:46.021876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13455332-9d6d-401b-9039-2ecb3a13ac80 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:46.040363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7eb4e6-6977-40b7-9051-35b40d81c2c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:46.041739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eb4e6-6977-40b7-9051-35b40d81c2c9 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:46.042320 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eb4e6-6977-40b7-9051-35b40d81c2c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:46.042846 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eb4e6-6977-40b7-9051-35b40d81c2c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:46.065483 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 851/1691] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:45 2022] POST /identity/v3/users => generated 282 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:46.071700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851d037-a801-4e75-9792-1bc68c59cd21 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:46.072063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851d037-a801-4e75-9792-1bc68c59cd21 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:46.072988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851d037-a801-4e75-9792-1bc68c59cd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:46.096467 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7eb4e6-6977-40b7-9051-35b40d81c2c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:46.098542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7eb4e6-6977-40b7-9051-35b40d81c2c9 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:46.113681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 841/1692] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:46.115534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7eb4e6-6977-40b7-9051-35b40d81c2c9 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:46.126533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a851d037-a801-4e75-9792-1bc68c59cd21 None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:46.128044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a851d037-a801-4e75-9792-1bc68c59cd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:46.141839 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7eb4e6-6977-40b7-9051-35b40d81c2c9 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:46.155585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72aafc-f422-4d80-8f3c-70cd776895b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:46.157126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72aafc-f422-4d80-8f3c-70cd776895b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:46.161020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72aafc-f422-4d80-8f3c-70cd776895b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:46.161412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72aafc-f422-4d80-8f3c-70cd776895b5 admin admin] PATH_INFO: `/v3/projects/5df94e79b6fb46da9ed7ddeb74bde5f4/users/074ab3f373da4cdfbeb33d19323b2012/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:46.207850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72aafc-f422-4d80-8f3c-70cd776895b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df94e79b6fb46da9ed7ddeb74bde5f4, user_id=074ab3f373da4cdfbeb33d19323b2012, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:46.216811 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72aafc-f422-4d80-8f3c-70cd776895b5 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:46.240284 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 842/1693] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:46 2022] PUT /identity/v3/projects/5df94e79b6fb46da9ed7ddeb74bde5f4/users/074ab3f373da4cdfbeb33d19323b2012/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:46.241755 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72aafc-f422-4d80-8f3c-70cd776895b5 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:46.282819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 852/1694] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:51:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:46.283575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851d037-a801-4e75-9792-1bc68c59cd21 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:46.336513 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851d037-a801-4e75-9792-1bc68c59cd21 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:46.440780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72aafc-f422-4d80-8f3c-70cd776895b5 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:46.456440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc6065-9886-4331-a576-e8032f3a5b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:46.457229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc6065-9886-4331-a576-e8032f3a5b39 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:46.457772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc6065-9886-4331-a576-e8032f3a5b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:46.459169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc6065-9886-4331-a576-e8032f3a5b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:46.460201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc6065-9886-4331-a576-e8032f3a5b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:46.462377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc6065-9886-4331-a576-e8032f3a5b39 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:46.469078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00dbcd-d36d-4d66-9ab0-9aaaabbc4728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:46.469906 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00dbcd-d36d-4d66-9ab0-9aaaabbc4728 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:46.470282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00dbcd-d36d-4d66-9ab0-9aaaabbc4728 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:46.470655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00dbcd-d36d-4d66-9ab0-9aaaabbc4728 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:46.471573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00dbcd-d36d-4d66-9ab0-9aaaabbc4728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:46.475492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00dbcd-d36d-4d66-9ab0-9aaaabbc4728 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:46.483709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 843/1695] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:46.488887 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc6065-9886-4331-a576-e8032f3a5b39 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:46.501286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 853/1696] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:46.505988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00dbcd-d36d-4d66-9ab0-9aaaabbc4728 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:46.507325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc6065-9886-4331-a576-e8032f3a5b39 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:46.521695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65335a7d-ec46-4ad8-9401-cd30ec6d15c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:46.523724 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335a7d-ec46-4ad8-9401-cd30ec6d15c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:46.524123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335a7d-ec46-4ad8-9401-cd30ec6d15c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:46.524298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00dbcd-d36d-4d66-9ab0-9aaaabbc4728 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:46.525296 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335a7d-ec46-4ad8-9401-cd30ec6d15c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:46.526257 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65335a7d-ec46-4ad8-9401-cd30ec6d15c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:46.530973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65335a7d-ec46-4ad8-9401-cd30ec6d15c1 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:46.539696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcea629-87f4-4459-94c3-1600a5efe05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:46.540765 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcea629-87f4-4459-94c3-1600a5efe05f admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:46.541148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcea629-87f4-4459-94c3-1600a5efe05f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:46.541520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcea629-87f4-4459-94c3-1600a5efe05f admin admin] PATH_INFO: `/v3/projects/5df94e79b6fb46da9ed7ddeb74bde5f4/users/074ab3f373da4cdfbeb33d19323b2012/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:46.546011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcea629-87f4-4459-94c3-1600a5efe05f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df94e79b6fb46da9ed7ddeb74bde5f4, user_id=074ab3f373da4cdfbeb33d19323b2012, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:46.548882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcea629-87f4-4459-94c3-1600a5efe05f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:46.555380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 844/1697] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:46.557113 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65335a7d-ec46-4ad8-9401-cd30ec6d15c1 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:46.573226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 854/1698] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:46 2022] PUT /identity/v3/projects/5df94e79b6fb46da9ed7ddeb74bde5f4/users/074ab3f373da4cdfbeb33d19323b2012/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:51:46.576075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65335a7d-ec46-4ad8-9401-cd30ec6d15c1 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:46.585903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44919b3b-8ebf-4276-9394-844e6457fbfc None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:46.586192 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44919b3b-8ebf-4276-9394-844e6457fbfc None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:46.586567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44919b3b-8ebf-4276-9394-844e6457fbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:46.661354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-44919b3b-8ebf-4276-9394-844e6457fbfc None None] MFA Rules not processed for user `074ab3f373da4cdfbeb33d19323b2012`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:46.665569 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-44919b3b-8ebf-4276-9394-844e6457fbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:46.744523 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf44ad-69f5-42a2-b4b6-66773103b579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:46.747694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf44ad-69f5-42a2-b4b6-66773103b579 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:46.747995 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf44ad-69f5-42a2-b4b6-66773103b579 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:46.748279 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf44ad-69f5-42a2-b4b6-66773103b579 admin admin] PATH_INFO: `/v3/projects/79e233fe1abb4beda8f9d9755c0482f9/users/a457fbf9357347349af2d1d8bd8b5ed8/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:46.783599 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf44ad-69f5-42a2-b4b6-66773103b579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e233fe1abb4beda8f9d9755c0482f9, user_id=a457fbf9357347349af2d1d8bd8b5ed8, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:46.789138 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf44ad-69f5-42a2-b4b6-66773103b579 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:46.832059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 845/1699] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:46 2022] PUT /identity/v3/projects/79e233fe1abb4beda8f9d9755c0482f9/users/a457fbf9357347349af2d1d8bd8b5ed8/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:46.834599 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf44ad-69f5-42a2-b4b6-66773103b579 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:46.839563 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-e5bf44ad-69f5-42a2-b4b6-66773103b579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:46.853326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 855/1700] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:46.857671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eff620-a54f-4c2a-9c76-d77ef8a69063 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:46.857984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eff620-a54f-4c2a-9c76-d77ef8a69063 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:46.858640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eff620-a54f-4c2a-9c76-d77ef8a69063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:46.918022 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-04eff620-a54f-4c2a-9c76-d77ef8a69063 None None] MFA Rules not processed for user `a457fbf9357347349af2d1d8bd8b5ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:46.920123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-04eff620-a54f-4c2a-9c76-d77ef8a69063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:46.974404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf44ad-69f5-42a2-b4b6-66773103b579 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:46.981650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db7465-6548-4323-8716-628715f82e72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:46.982612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db7465-6548-4323-8716-628715f82e72 service swift] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:46.983075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db7465-6548-4323-8716-628715f82e72 service swift] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:46.983486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db7465-6548-4323-8716-628715f82e72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:47.005025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 856/1701] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:46 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:47.005957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eff620-a54f-4c2a-9c76-d77ef8a69063 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:47.015573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-04eff620-a54f-4c2a-9c76-d77ef8a69063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:47.063863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db7465-6548-4323-8716-628715f82e72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:47.067774 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db7465-6548-4323-8716-628715f82e72 service swift] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:47.089925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 846/1702] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:51:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:51:47.097788 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db7465-6548-4323-8716-628715f82e72 service swift] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:47.114865 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eff620-a54f-4c2a-9c76-d77ef8a69063 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:47.121855 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17774b-b32d-4e0a-9bb6-c734001fc03d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:47.122721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17774b-b32d-4e0a-9bb6-c734001fc03d service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:47.123150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17774b-b32d-4e0a-9bb6-c734001fc03d service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:47.123736 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17774b-b32d-4e0a-9bb6-c734001fc03d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:47.265806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17774b-b32d-4e0a-9bb6-c734001fc03d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:47.267866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17774b-b32d-4e0a-9bb6-c734001fc03d service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:47.293392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 857/1703] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:47.305722 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db7465-6548-4323-8716-628715f82e72 service swift] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:47.318999 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba690bf6-6779-4361-b7a8-849ccb3895aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:47.320251 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba690bf6-6779-4361-b7a8-849ccb3895aa admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:47.322219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba690bf6-6779-4361-b7a8-849ccb3895aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:47.322517 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba690bf6-6779-4361-b7a8-849ccb3895aa admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:47.323326 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba690bf6-6779-4361-b7a8-849ccb3895aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:47.330334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba690bf6-6779-4361-b7a8-849ccb3895aa admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:47.438536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17774b-b32d-4e0a-9bb6-c734001fc03d service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:47.442592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 847/1704] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:47 2022] POST /identity/v3/projects => generated 365 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:47.444418 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba690bf6-6779-4361-b7a8-849ccb3895aa admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:47.455843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17774b-b32d-4e0a-9bb6-c734001fc03d service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:47.614692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba690bf6-6779-4361-b7a8-849ccb3895aa admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:47.632239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731132a-db61-454f-ad6e-d173891d7fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:47.633491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731132a-db61-454f-ad6e-d173891d7fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:47.634031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731132a-db61-454f-ad6e-d173891d7fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:47.634524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731132a-db61-454f-ad6e-d173891d7fc6 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:47.644075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d08fa0-9f6e-4238-9ff3-116c93a4de2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:47.644779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d08fa0-9f6e-4238-9ff3-116c93a4de2a service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:47.644970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d08fa0-9f6e-4238-9ff3-116c93a4de2a service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:47.645422 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d08fa0-9f6e-4238-9ff3-116c93a4de2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:47.646611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4731132a-db61-454f-ad6e-d173891d7fc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:47.649462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4731132a-db61-454f-ad6e-d173891d7fc6 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:47.652170 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-78d08fa0-9f6e-4238-9ff3-116c93a4de2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:47.720278 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 848/1705] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:47 2022] POST /identity/v3/users => generated 289 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:47.735349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731132a-db61-454f-ad6e-d173891d7fc6 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:47.786612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731132a-db61-454f-ad6e-d173891d7fc6 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:47.806017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a34b16-2aa6-4ec4-996d-e932d3ccfe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:47.809080 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a34b16-2aa6-4ec4-996d-e932d3ccfe7b admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:47.812937 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a34b16-2aa6-4ec4-996d-e932d3ccfe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:47.813278 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a34b16-2aa6-4ec4-996d-e932d3ccfe7b admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:47.815031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a34b16-2aa6-4ec4-996d-e932d3ccfe7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:47.821779 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a34b16-2aa6-4ec4-996d-e932d3ccfe7b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:47.860431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 849/1706] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:47.862669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a34b16-2aa6-4ec4-996d-e932d3ccfe7b admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:47.864610 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-59a34b16-2aa6-4ec4-996d-e932d3ccfe7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:47.874494 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d08fa0-9f6e-4238-9ff3-116c93a4de2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:47.879907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d08fa0-9f6e-4238-9ff3-116c93a4de2a service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:47.908771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a34b16-2aa6-4ec4-996d-e932d3ccfe7b admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:47.918173 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 858/1707] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:47.926265 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eca30f-dc15-4496-bbac-5922b2504a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:47.927452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eca30f-dc15-4496-bbac-5922b2504a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:47.928031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eca30f-dc15-4496-bbac-5922b2504a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:47.928139 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d08fa0-9f6e-4238-9ff3-116c93a4de2a service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:47.928543 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eca30f-dc15-4496-bbac-5922b2504a7e admin admin] PATH_INFO: `/v3/users/e456da1e33c14bafa22f975abfc905bc` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:47.930394 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eca30f-dc15-4496-bbac-5922b2504a7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e456da1e33c14bafa22f975abfc905bc)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:47.935330 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eca30f-dc15-4496-bbac-5922b2504a7e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:47.987912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d08fa0-9f6e-4238-9ff3-116c93a4de2a service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:48.011701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f04172-d156-4aee-8062-128149582113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:48.015025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f04172-d156-4aee-8062-128149582113 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:48.015350 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f04172-d156-4aee-8062-128149582113 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:48.019085 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f04172-d156-4aee-8062-128149582113 admin admin] PATH_INFO: `/v3/projects/04e0137f927943859f6a355b37be9dd0/users/1544a91ba9d54b859e35a92cfcf3cdb9/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:48.052796 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-38eca30f-dc15-4496-bbac-5922b2504a7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e456da1e33c14bafa22f975abfc905bc'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:48.080759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-38eca30f-dc15-4496-bbac-5922b2504a7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e456da1e33c14bafa22f975abfc905bc'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:48.085547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-38eca30f-dc15-4496-bbac-5922b2504a7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e456da1e33c14bafa22f975abfc905bc'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:48.086432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f04172-d156-4aee-8062-128149582113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e0137f927943859f6a355b37be9dd0, user_id=1544a91ba9d54b859e35a92cfcf3cdb9, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:48.087019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-38eca30f-dc15-4496-bbac-5922b2504a7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e456da1e33c14bafa22f975abfc905bc'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:48.098759 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f04172-d156-4aee-8062-128149582113 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:48.106894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-38eca30f-dc15-4496-bbac-5922b2504a7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e456da1e33c14bafa22f975abfc905bc'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:48.139220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 850/1708] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:51:47 2022] DELETE /identity/v3/users/e456da1e33c14bafa22f975abfc905bc => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:48.159197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba50d7-1898-4821-b95d-a36f0390ac2e None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:48.159663 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba50d7-1898-4821-b95d-a36f0390ac2e None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:48.160087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba50d7-1898-4821-b95d-a36f0390ac2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:48.171053 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 859/1709] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:47 2022] PUT /identity/v3/projects/04e0137f927943859f6a355b37be9dd0/users/1544a91ba9d54b859e35a92cfcf3cdb9/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:48.175811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f04172-d156-4aee-8062-128149582113 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:48.185587 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a7f04172-d156-4aee-8062-128149582113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:48.204222 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-26ba50d7-1898-4821-b95d-a36f0390ac2e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:48.206753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-26ba50d7-1898-4821-b95d-a36f0390ac2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:48.454138 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f04172-d156-4aee-8062-128149582113 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:48.509840 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 851/1710] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:48.588188 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba50d7-1898-4821-b95d-a36f0390ac2e None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:48.589987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-26ba50d7-1898-4821-b95d-a36f0390ac2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:48.741079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aac520-495e-4b0a-aa05-1fe7192a6593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:48.742107 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aac520-495e-4b0a-aa05-1fe7192a6593 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:48.742393 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aac520-495e-4b0a-aa05-1fe7192a6593 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:48.744609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aac520-495e-4b0a-aa05-1fe7192a6593 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:48.745361 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aac520-495e-4b0a-aa05-1fe7192a6593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:48.747674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aac520-495e-4b0a-aa05-1fe7192a6593 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:48.779988 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba50d7-1898-4821-b95d-a36f0390ac2e None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:48.780194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 860/1711] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 605 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:48.781495 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aac520-495e-4b0a-aa05-1fe7192a6593 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:48.791904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca4167-f76e-4101-a06d-6405bcc1bbb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:48.792595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca4167-f76e-4101-a06d-6405bcc1bbb7 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:48.792885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca4167-f76e-4101-a06d-6405bcc1bbb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:48.793168 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca4167-f76e-4101-a06d-6405bcc1bbb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:48.814926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aac520-495e-4b0a-aa05-1fe7192a6593 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:48.821500 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca4167-f76e-4101-a06d-6405bcc1bbb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:48.823168 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca4167-f76e-4101-a06d-6405bcc1bbb7 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:48.831127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b3720-9fb9-4286-95f9-6cbf703f93a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:48.832038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3720-9fb9-4286-95f9-6cbf703f93a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:48.832397 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3720-9fb9-4286-95f9-6cbf703f93a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:48.832811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3720-9fb9-4286-95f9-6cbf703f93a2 admin admin] PATH_INFO: `/v3/projects/04e0137f927943859f6a355b37be9dd0/users/1544a91ba9d54b859e35a92cfcf3cdb9/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:48.836100 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b3720-9fb9-4286-95f9-6cbf703f93a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e0137f927943859f6a355b37be9dd0, user_id=1544a91ba9d54b859e35a92cfcf3cdb9, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:48.839545 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b3720-9fb9-4286-95f9-6cbf703f93a2 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:48.850588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 852/1712] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:48.870852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 861/1713] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:48 2022] PUT /identity/v3/projects/04e0137f927943859f6a355b37be9dd0/users/1544a91ba9d54b859e35a92cfcf3cdb9/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:48.871061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca4167-f76e-4101-a06d-6405bcc1bbb7 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:48.871061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-72ca4167-f76e-4101-a06d-6405bcc1bbb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:48.882439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0b6e6-852e-4915-b946-bdbb4c5ef147 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:48.882754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0b6e6-852e-4915-b946-bdbb4c5ef147 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:48.890691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0b6e6-852e-4915-b946-bdbb4c5ef147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:48.914446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca4167-f76e-4101-a06d-6405bcc1bbb7 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:48.995690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-1dc0b6e6-852e-4915-b946-bdbb4c5ef147 None None] MFA Rules not processed for user `1544a91ba9d54b859e35a92cfcf3cdb9`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:48.999014 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-1dc0b6e6-852e-4915-b946-bdbb4c5ef147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:49.080002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31385b95-cc4b-451d-9670-580bea7ff622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:49.082504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31385b95-cc4b-451d-9670-580bea7ff622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:49.082504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31385b95-cc4b-451d-9670-580bea7ff622 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:49.082811 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31385b95-cc4b-451d-9670-580bea7ff622 admin admin] PATH_INFO: `/v3/users/271827a6cf9344318a279d4daf6e4a09` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:49.090991 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31385b95-cc4b-451d-9670-580bea7ff622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=271827a6cf9344318a279d4daf6e4a09)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:49.093425 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31385b95-cc4b-451d-9670-580bea7ff622 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:49.176384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 862/1714] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:48 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:49.194620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca006ef-f99f-4370-9692-cdfe2bad77c3 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:49.196316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca006ef-f99f-4370-9692-cdfe2bad77c3 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:49.199065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca006ef-f99f-4370-9692-cdfe2bad77c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:49.220924 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-31385b95-cc4b-451d-9670-580bea7ff622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '271827a6cf9344318a279d4daf6e4a09'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:49.235340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7ca006ef-f99f-4370-9692-cdfe2bad77c3 None None] MFA Rules not processed for user `c4df058606c448a1bb757020e869ac33`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:49.236920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7ca006ef-f99f-4370-9692-cdfe2bad77c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:49.244501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-31385b95-cc4b-451d-9670-580bea7ff622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '271827a6cf9344318a279d4daf6e4a09'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:49.255479 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-31385b95-cc4b-451d-9670-580bea7ff622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '271827a6cf9344318a279d4daf6e4a09'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:49.256312 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-31385b95-cc4b-451d-9670-580bea7ff622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '271827a6cf9344318a279d4daf6e4a09'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:49.275659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-31385b95-cc4b-451d-9670-580bea7ff622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '271827a6cf9344318a279d4daf6e4a09'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:49.293086 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 853/1715] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:51:48 2022] DELETE /identity/v3/users/271827a6cf9344318a279d4daf6e4a09 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:51:49.304911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31385b95-cc4b-451d-9670-580bea7ff622 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:49.308143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-31385b95-cc4b-451d-9670-580bea7ff622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:49.383952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 863/1716] 10.222.0.58 () {58 vars in 1015 bytes} [Fri May 13 21:51:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:49.409929 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca006ef-f99f-4370-9692-cdfe2bad77c3 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:49.409929 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7ca006ef-f99f-4370-9692-cdfe2bad77c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:49.433732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca006ef-f99f-4370-9692-cdfe2bad77c3 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:49.499679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31385b95-cc4b-451d-9670-580bea7ff622 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:49.509101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21b60b-fe97-4aed-a4e1-a2c099d90dcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:49.510100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21b60b-fe97-4aed-a4e1-a2c099d90dcb service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:49.510762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21b60b-fe97-4aed-a4e1-a2c099d90dcb service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:49.510762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21b60b-fe97-4aed-a4e1-a2c099d90dcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:49.516417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-3b21b60b-fe97-4aed-a4e1-a2c099d90dcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:49.597060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40112759-5884-4378-bd3f-15daa03887c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:49.597874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40112759-5884-4378-bd3f-15daa03887c8 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:49.598279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40112759-5884-4378-bd3f-15daa03887c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:49.598660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40112759-5884-4378-bd3f-15daa03887c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:49.623640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40112759-5884-4378-bd3f-15daa03887c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:49.635688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40112759-5884-4378-bd3f-15daa03887c8 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:49.671795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 864/1717] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:51:49.759701 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21b60b-fe97-4aed-a4e1-a2c099d90dcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:49.761237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21b60b-fe97-4aed-a4e1-a2c099d90dcb service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:49.796733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 854/1718] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:49 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:50.007272 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40112759-5884-4378-bd3f-15daa03887c8 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:50.018299 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40112759-5884-4378-bd3f-15daa03887c8 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:50.039671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930a798-3eb5-4086-a011-fefdc22800e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:50.040857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930a798-3eb5-4086-a011-fefdc22800e6 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:50.041355 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930a798-3eb5-4086-a011-fefdc22800e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:50.042026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930a798-3eb5-4086-a011-fefdc22800e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:50.044705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b930a798-3eb5-4086-a011-fefdc22800e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:50.181486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930a798-3eb5-4086-a011-fefdc22800e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:50.186105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930a798-3eb5-4086-a011-fefdc22800e6 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:50.263210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 865/1719] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:50 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:50.266456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21b60b-fe97-4aed-a4e1-a2c099d90dcb service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:50.268021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-3b21b60b-fe97-4aed-a4e1-a2c099d90dcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:50.456049 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21b60b-fe97-4aed-a4e1-a2c099d90dcb service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:50.472660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:50.476792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:50.477250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:50.477723 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] PATH_INFO: `/v3/projects/5b7079f32018466e99a74b43dbb0a5f8` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:50.483074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7079f32018466e99a74b43dbb0a5f8)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:50.487494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:50.639755 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7079f32018466e99a74b43dbb0a5f8', 'request_id': 'req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:50.665286 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7079f32018466e99a74b43dbb0a5f8', 'request_id': 'req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:50.681211 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7079f32018466e99a74b43dbb0a5f8', 'request_id': 'req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:50.710950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] The token cache is being invalidate because project 5b7079f32018466e99a74b43dbb0a5f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:51:50.711294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:50.712104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:50.714486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 855/1720] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:51:50 2022] DELETE /identity/v3/projects/5b7079f32018466e99a74b43dbb0a5f8 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:50.718347 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930a798-3eb5-4086-a011-fefdc22800e6 service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:50.734209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930a798-3eb5-4086-a011-fefdc22800e6 service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:50.917933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:50.919741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:50.963442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824131b-98bb-440e-8e69-523eeffb24b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:50.964501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824131b-98bb-440e-8e69-523eeffb24b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:50.964789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824131b-98bb-440e-8e69-523eeffb24b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:50.965508 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824131b-98bb-440e-8e69-523eeffb24b7 admin admin] PATH_INFO: `/v3/users/8e28e4fb3479472c9f831354bc529ccc` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:50.968632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824131b-98bb-440e-8e69-523eeffb24b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e28e4fb3479472c9f831354bc529ccc)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:50.973611 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824131b-98bb-440e-8e69-523eeffb24b7 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:51.105823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ee47f-b5d2-47aa-8671-b3bbd4f2d831 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:51.109274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7824131b-98bb-440e-8e69-523eeffb24b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e28e4fb3479472c9f831354bc529ccc'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:51.120686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabe434-18fa-4afd-adee-f3d9be8ada80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:51.122126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabe434-18fa-4afd-adee-f3d9be8ada80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:51.122453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabe434-18fa-4afd-adee-f3d9be8ada80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:51.122760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabe434-18fa-4afd-adee-f3d9be8ada80 admin admin] PATH_INFO: `/v3/projects/51ed5e55a27c4adca6394a55b5cbd7aa` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:51.136262 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabe434-18fa-4afd-adee-f3d9be8ada80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51ed5e55a27c4adca6394a55b5cbd7aa)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:51.144405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabe434-18fa-4afd-adee-f3d9be8ada80 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:51.162428 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7824131b-98bb-440e-8e69-523eeffb24b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e28e4fb3479472c9f831354bc529ccc'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:51.165992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7824131b-98bb-440e-8e69-523eeffb24b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e28e4fb3479472c9f831354bc529ccc'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:51.167090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7824131b-98bb-440e-8e69-523eeffb24b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e28e4fb3479472c9f831354bc529ccc'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:51.180863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7824131b-98bb-440e-8e69-523eeffb24b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e28e4fb3479472c9f831354bc529ccc'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:51.210656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 866/1721] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:51:50 2022] DELETE /identity/v3/users/8e28e4fb3479472c9f831354bc529ccc => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:51.301768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bfabe434-18fa-4afd-adee-f3d9be8ada80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51ed5e55a27c4adca6394a55b5cbd7aa', 'request_id': 'req-bfabe434-18fa-4afd-adee-f3d9be8ada80'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:51.311839 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bfabe434-18fa-4afd-adee-f3d9be8ada80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51ed5e55a27c4adca6394a55b5cbd7aa', 'request_id': 'req-bfabe434-18fa-4afd-adee-f3d9be8ada80'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:51.334294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bfabe434-18fa-4afd-adee-f3d9be8ada80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51ed5e55a27c4adca6394a55b5cbd7aa', 'request_id': 'req-bfabe434-18fa-4afd-adee-f3d9be8ada80'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:51.369785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bfabe434-18fa-4afd-adee-f3d9be8ada80 admin admin] The token cache is being invalidate because project 51ed5e55a27c4adca6394a55b5cbd7aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:51:51.370083 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bfabe434-18fa-4afd-adee-f3d9be8ada80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:51.373651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bfabe434-18fa-4afd-adee-f3d9be8ada80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:51.381633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 856/1722] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:51:50 2022] DELETE /identity/v3/projects/51ed5e55a27c4adca6394a55b5cbd7aa => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:51:51.382415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163fdfe-7ba2-40cb-a99e-0a6e7c0e2546 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:51.382903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163fdfe-7ba2-40cb-a99e-0a6e7c0e2546 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:51.383314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163fdfe-7ba2-40cb-a99e-0a6e7c0e2546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:51.431209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-b163fdfe-7ba2-40cb-a99e-0a6e7c0e2546 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:51.432425 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b163fdfe-7ba2-40cb-a99e-0a6e7c0e2546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:51.541530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 867/1723] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:51:51.552537 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70aa7b-2072-4bb2-a3fa-d0e1164e176a None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:51.552842 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70aa7b-2072-4bb2-a3fa-d0e1164e176a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:51.553217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70aa7b-2072-4bb2-a3fa-d0e1164e176a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:51.612653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-2c70aa7b-2072-4bb2-a3fa-d0e1164e176a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:51.623875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2c70aa7b-2072-4bb2-a3fa-d0e1164e176a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:51.657614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 857/1724] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:51.661652 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163fdfe-7ba2-40cb-a99e-0a6e7c0e2546 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:51.679801 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163fdfe-7ba2-40cb-a99e-0a6e7c0e2546 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:51.701481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b927f76-ab84-4747-880c-dbbee4729833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:51.703277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b927f76-ab84-4747-880c-dbbee4729833 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:51.704157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b927f76-ab84-4747-880c-dbbee4729833 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:51.705241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b927f76-ab84-4747-880c-dbbee4729833 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:51.719737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b927f76-ab84-4747-880c-dbbee4729833 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:51:51.723220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b927f76-ab84-4747-880c-dbbee4729833 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:51.725046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b927f76-ab84-4747-880c-dbbee4729833 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:51.745208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 868/1725] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:51:51.750170 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70aa7b-2072-4bb2-a3fa-d0e1164e176a None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:51.775054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70aa7b-2072-4bb2-a3fa-d0e1164e176a None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:51.781730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc6cd4-f3c9-4625-8e0c-84f7808d83f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:51.782822 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc6cd4-f3c9-4625-8e0c-84f7808d83f2 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:51.783309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc6cd4-f3c9-4625-8e0c-84f7808d83f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:51.783801 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc6cd4-f3c9-4625-8e0c-84f7808d83f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:51.785512 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc6cd4-f3c9-4625-8e0c-84f7808d83f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:51.788351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc6cd4-f3c9-4625-8e0c-84f7808d83f2 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:51.892256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 858/1726] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:51 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:51.896890 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b927f76-ab84-4747-880c-dbbee4729833 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:51.921022 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b927f76-ab84-4747-880c-dbbee4729833 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:52.136916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bb4bd-38f7-4abb-b425-edea984fb61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:52.138148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bb4bd-38f7-4abb-b425-edea984fb61d admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:52.138739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bb4bd-38f7-4abb-b425-edea984fb61d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:52.139228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bb4bd-38f7-4abb-b425-edea984fb61d admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:52.140159 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bb4bd-38f7-4abb-b425-edea984fb61d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:52.145968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bb4bd-38f7-4abb-b425-edea984fb61d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:52.198922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 869/1727] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:51 2022] POST /identity/v3/users => generated 283 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:51:52.206239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc6cd4-f3c9-4625-8e0c-84f7808d83f2 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:52.219774 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc6cd4-f3c9-4625-8e0c-84f7808d83f2 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:52.226845 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57912c36-17d0-4628-bd47-b65027487a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:52.227995 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57912c36-17d0-4628-bd47-b65027487a08 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:52.228434 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57912c36-17d0-4628-bd47-b65027487a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:52.228821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57912c36-17d0-4628-bd47-b65027487a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:52.229670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57912c36-17d0-4628-bd47-b65027487a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:52.231973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57912c36-17d0-4628-bd47-b65027487a08 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:52.238832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28af66-bcdd-458e-9e53-2e8de3de983a None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:52.239381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28af66-bcdd-458e-9e53-2e8de3de983a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:52.239381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28af66-bcdd-458e-9e53-2e8de3de983a None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:52.241671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 870/1728] 10.222.0.35 () {54 vars in 921 bytes} [Fri May 13 21:51:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:51:52.249407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 859/1729] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:52.253831 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28af66-bcdd-458e-9e53-2e8de3de983a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:52.277523 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a016-248b-4fac-8f86-e00d9301d181 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:52.278996 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a016-248b-4fac-8f86-e00d9301d181 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:52.280053 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a016-248b-4fac-8f86-e00d9301d181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:52.285965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28af66-bcdd-458e-9e53-2e8de3de983a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:52.299023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef7f60-99bb-43f3-9e1c-97d8fa99603f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:52.299719 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef7f60-99bb-43f3-9e1c-97d8fa99603f admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:52.300060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef7f60-99bb-43f3-9e1c-97d8fa99603f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:52.300393 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef7f60-99bb-43f3-9e1c-97d8fa99603f admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:52.301136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef7f60-99bb-43f3-9e1c-97d8fa99603f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:52.303779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef7f60-99bb-43f3-9e1c-97d8fa99603f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:52.334085 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-7b56a016-248b-4fac-8f86-e00d9301d181 None None] MFA Rules not processed for user `316ef43c6c304bd3a27e085bf4da8a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:52.337275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 871/1730] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:52.337275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef7f60-99bb-43f3-9e1c-97d8fa99603f admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:52.337477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-78ef7f60-99bb-43f3-9e1c-97d8fa99603f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:52.338592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-7b56a016-248b-4fac-8f86-e00d9301d181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:52.371210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef7f60-99bb-43f3-9e1c-97d8fa99603f admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:52.386834 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:52.388266 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:52.388556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:52.388837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] PATH_INFO: `/v3/projects/cd76838c6c91427f8b0331f5338afeff` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:52.390360 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd76838c6c91427f8b0331f5338afeff)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:52.392385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:52.558092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 860/1731] 10.222.0.35 () {58 vars in 1018 bytes} [Fri May 13 21:51:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:52.575911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56a016-248b-4fac-8f86-e00d9301d181 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:52.577714 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd76838c6c91427f8b0331f5338afeff', 'request_id': 'req-bf76cb89-5e61-4887-b7d8-ad1b67586f85'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:52.593067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56a016-248b-4fac-8f86-e00d9301d181 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:52.597593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd76838c6c91427f8b0331f5338afeff', 'request_id': 'req-bf76cb89-5e61-4887-b7d8-ad1b67586f85'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:52.625432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd76838c6c91427f8b0331f5338afeff', 'request_id': 'req-bf76cb89-5e61-4887-b7d8-ad1b67586f85'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:52.670875 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] The token cache is being invalidate because project cd76838c6c91427f8b0331f5338afeff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:51:52.671183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:52.678262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:52.683602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 872/1732] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:51:52 2022] DELETE /identity/v3/projects/cd76838c6c91427f8b0331f5338afeff => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:52.685126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:52.692042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:52.758550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7a8b5-4123-4c1a-ae83-d1115f3178d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:52.768237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7a8b5-4123-4c1a-ae83-d1115f3178d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:52.768530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7a8b5-4123-4c1a-ae83-d1115f3178d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:52.768808 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7a8b5-4123-4c1a-ae83-d1115f3178d4 admin admin] PATH_INFO: `/v3/projects/8d52b78bce534d219ac697c03c95d230/users/5c3fcbe716c74090b101a068e1ab2ec7/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:52.809959 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7a8b5-4123-4c1a-ae83-d1115f3178d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d52b78bce534d219ac697c03c95d230, user_id=5c3fcbe716c74090b101a068e1ab2ec7, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:52.813237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7a8b5-4123-4c1a-ae83-d1115f3178d4 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:52.850696 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 861/1733] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:52 2022] PUT /identity/v3/projects/8d52b78bce534d219ac697c03c95d230/users/5c3fcbe716c74090b101a068e1ab2ec7/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:52.850696 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee720899-84b1-49fa-a340-8be6aa307a2d None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:52.850696 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee720899-84b1-49fa-a340-8be6aa307a2d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:52.851068 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee720899-84b1-49fa-a340-8be6aa307a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:52.881768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ee720899-84b1-49fa-a340-8be6aa307a2d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:52.883710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ee720899-84b1-49fa-a340-8be6aa307a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:52.908130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76cb89-5e61-4887-b7d8-ad1b67586f85 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:52.975323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 862/1734] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:52.978729 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eade6ee-33ce-4b9b-b2e6-455686d72d51 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:52.979294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eade6ee-33ce-4b9b-b2e6-455686d72d51 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:52.979879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eade6ee-33ce-4b9b-b2e6-455686d72d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.025300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-7eade6ee-33ce-4b9b-b2e6-455686d72d51 None None] MFA Rules not processed for user `5c3fcbe716c74090b101a068e1ab2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:53.026289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-7eade6ee-33ce-4b9b-b2e6-455686d72d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:53.059631 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765b5b1-4fd8-4081-86ec-3c3151b1afc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:53.063004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765b5b1-4fd8-4081-86ec-3c3151b1afc9 service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:53.067664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765b5b1-4fd8-4081-86ec-3c3151b1afc9 service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:53.068255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765b5b1-4fd8-4081-86ec-3c3151b1afc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.073390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-5765b5b1-4fd8-4081-86ec-3c3151b1afc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:53.111771 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765b5b1-4fd8-4081-86ec-3c3151b1afc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:53.114036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765b5b1-4fd8-4081-86ec-3c3151b1afc9 service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:53.129357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 873/1735] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:53.131326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19d87d-e9cb-4f3a-856c-ee0c8988b0c2 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:53.131881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19d87d-e9cb-4f3a-856c-ee0c8988b0c2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:53.132408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19d87d-e9cb-4f3a-856c-ee0c8988b0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.156619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-4f19d87d-e9cb-4f3a-856c-ee0c8988b0c2 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:53.158282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-4f19d87d-e9cb-4f3a-856c-ee0c8988b0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:53.170258 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 863/1736] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:53.185291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eade6ee-33ce-4b9b-b2e6-455686d72d51 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:53.198351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-7eade6ee-33ce-4b9b-b2e6-455686d72d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:53.206021 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 874/1737] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:53.208918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19d87d-e9cb-4f3a-856c-ee0c8988b0c2 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:53.233820 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19d87d-e9cb-4f3a-856c-ee0c8988b0c2 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:53.257467 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d848c-7765-4df9-a128-34a7770eb931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:53.263101 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d848c-7765-4df9-a128-34a7770eb931 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:53.265027 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d848c-7765-4df9-a128-34a7770eb931 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:53.265394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d848c-7765-4df9-a128-34a7770eb931 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.266127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d848c-7765-4df9-a128-34a7770eb931 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:51:53.266442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d848c-7765-4df9-a128-34a7770eb931 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:53.274192 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d848c-7765-4df9-a128-34a7770eb931 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:53.293026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 875/1738] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:51:53.297747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d848c-7765-4df9-a128-34a7770eb931 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:53.308935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d848c-7765-4df9-a128-34a7770eb931 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:53.316302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295346ba-903b-4376-b68c-4ed60d74c407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:53.317947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295346ba-903b-4376-b68c-4ed60d74c407 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:53.318385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295346ba-903b-4376-b68c-4ed60d74c407 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:53.319034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295346ba-903b-4376-b68c-4ed60d74c407 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.321045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295346ba-903b-4376-b68c-4ed60d74c407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:53.323449 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295346ba-903b-4376-b68c-4ed60d74c407 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:53.359342 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eade6ee-33ce-4b9b-b2e6-455686d72d51 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:53.363664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 876/1739] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:53 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:53.364774 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c19dff-8674-43fb-9aee-813f7f837f0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:53.365635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295346ba-903b-4376-b68c-4ed60d74c407 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:53.365792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c19dff-8674-43fb-9aee-813f7f837f0a service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:53.366151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c19dff-8674-43fb-9aee-813f7f837f0a service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:53.366877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c19dff-8674-43fb-9aee-813f7f837f0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.368469 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-295346ba-903b-4376-b68c-4ed60d74c407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:53.379857 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c19dff-8674-43fb-9aee-813f7f837f0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:53.382281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c19dff-8674-43fb-9aee-813f7f837f0a service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:53.395578 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 864/1740] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:53 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:53.396426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c19dff-8674-43fb-9aee-813f7f837f0a service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:53.410141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c19dff-8674-43fb-9aee-813f7f837f0a service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:53.453091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295346ba-903b-4376-b68c-4ed60d74c407 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:53.467842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6f655-34e7-4cca-9c20-0f1c517bc703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:53.469539 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6f655-34e7-4cca-9c20-0f1c517bc703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:53.469833 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6f655-34e7-4cca-9c20-0f1c517bc703 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:53.470113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6f655-34e7-4cca-9c20-0f1c517bc703 admin admin] PATH_INFO: `/v3/projects/14b40140d4ca4ee4b199e776dcfb652e` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.471712 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6f655-34e7-4cca-9c20-0f1c517bc703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14b40140d4ca4ee4b199e776dcfb652e)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:53.473808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6f655-34e7-4cca-9c20-0f1c517bc703 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:53.476384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6662ee-9d61-4305-abb1-2047cb9de297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:53.477193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6662ee-9d61-4305-abb1-2047cb9de297 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:53.477603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6662ee-9d61-4305-abb1-2047cb9de297 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:53.478011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6662ee-9d61-4305-abb1-2047cb9de297 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.480025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6662ee-9d61-4305-abb1-2047cb9de297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:53.483485 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6662ee-9d61-4305-abb1-2047cb9de297 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:53.512335 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 865/1741] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:53 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:53.516131 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6662ee-9d61-4305-abb1-2047cb9de297 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:53.531097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6662ee-9d61-4305-abb1-2047cb9de297 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:53.538368 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d99fe-ee76-414a-8fd2-dadefb3527e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:53.539234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d99fe-ee76-414a-8fd2-dadefb3527e2 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:53.539730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d99fe-ee76-414a-8fd2-dadefb3527e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:53.540131 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d99fe-ee76-414a-8fd2-dadefb3527e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.540971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d99fe-ee76-414a-8fd2-dadefb3527e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:53.542687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d99fe-ee76-414a-8fd2-dadefb3527e2 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:53.551669 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bdd6f655-34e7-4cca-9c20-0f1c517bc703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14b40140d4ca4ee4b199e776dcfb652e', 'request_id': 'req-bdd6f655-34e7-4cca-9c20-0f1c517bc703'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:53.555096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 866/1742] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:51:53.558890 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d99fe-ee76-414a-8fd2-dadefb3527e2 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:53.565017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bdd6f655-34e7-4cca-9c20-0f1c517bc703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14b40140d4ca4ee4b199e776dcfb652e', 'request_id': 'req-bdd6f655-34e7-4cca-9c20-0f1c517bc703'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:53.571207 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d99fe-ee76-414a-8fd2-dadefb3527e2 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:53.575703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bdd6f655-34e7-4cca-9c20-0f1c517bc703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14b40140d4ca4ee4b199e776dcfb652e', 'request_id': 'req-bdd6f655-34e7-4cca-9c20-0f1c517bc703'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:53.590121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bdd6f655-34e7-4cca-9c20-0f1c517bc703 admin admin] The token cache is being invalidate because project 14b40140d4ca4ee4b199e776dcfb652e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:51:53.591217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bdd6f655-34e7-4cca-9c20-0f1c517bc703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:53.592163 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bdd6f655-34e7-4cca-9c20-0f1c517bc703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:51:53.595054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 877/1743] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:51:53 2022] DELETE /identity/v3/projects/14b40140d4ca4ee4b199e776dcfb652e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:51:53.600249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0994dcd-0e6c-4acd-9366-16b0524f54ed None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:53.600769 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0994dcd-0e6c-4acd-9366-16b0524f54ed None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:53.601186 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0994dcd-0e6c-4acd-9366-16b0524f54ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.636501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-b0994dcd-0e6c-4acd-9366-16b0524f54ed None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:53.638113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b0994dcd-0e6c-4acd-9366-16b0524f54ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:53.664964 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2028da-d64a-48b1-a28c-9678f46a1d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:53.667537 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2028da-d64a-48b1-a28c-9678f46a1d68 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:53.670223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2028da-d64a-48b1-a28c-9678f46a1d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:53.672831 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2028da-d64a-48b1-a28c-9678f46a1d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.674123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2028da-d64a-48b1-a28c-9678f46a1d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:53.677651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2028da-d64a-48b1-a28c-9678f46a1d68 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:53.699270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 867/1744] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:53.701824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec187aeb-9189-4b01-96b2-43685d113be7 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:53.702340 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec187aeb-9189-4b01-96b2-43685d113be7 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:53.702941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec187aeb-9189-4b01-96b2-43685d113be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.765457 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ec187aeb-9189-4b01-96b2-43685d113be7 None None] MFA Rules not processed for user `0a1425d092844a2e9b8a917dc9981b32`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:53.767609 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ec187aeb-9189-4b01-96b2-43685d113be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:53.770574 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 878/1745] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:53.771799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0994dcd-0e6c-4acd-9366-16b0524f54ed None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:53.773805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b0994dcd-0e6c-4acd-9366-16b0524f54ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:53.788369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0994dcd-0e6c-4acd-9366-16b0524f54ed None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:53.797945 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8936b-d2c4-4ded-a935-c836c609602b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:53.799131 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8936b-d2c4-4ded-a935-c836c609602b admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:53.799520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8936b-d2c4-4ded-a935-c836c609602b admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:53.800176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8936b-d2c4-4ded-a935-c836c609602b admin admin] PATH_INFO: `/v3/projects/6b17a044889b481cad63732c38a1d89a/users/3c2857e783314386960a9970f4854f75/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.849140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8936b-d2c4-4ded-a935-c836c609602b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b17a044889b481cad63732c38a1d89a, user_id=3c2857e783314386960a9970f4854f75, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:53.852146 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8936b-d2c4-4ded-a935-c836c609602b admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:53.882575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 879/1746] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:53 2022] PUT /identity/v3/projects/6b17a044889b481cad63732c38a1d89a/users/3c2857e783314386960a9970f4854f75/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:53.884241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28756a2c-c22d-45ad-bf86-8fca3d1ac184 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:53.885148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28756a2c-c22d-45ad-bf86-8fca3d1ac184 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:53.886045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28756a2c-c22d-45ad-bf86-8fca3d1ac184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.915628 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-28756a2c-c22d-45ad-bf86-8fca3d1ac184 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:53.917180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-28756a2c-c22d-45ad-bf86-8fca3d1ac184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:53.918418 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 868/1747] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:53.920173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2fc8b-b0a7-4358-b2d5-8fc34f2b3b80 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:53.920739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2fc8b-b0a7-4358-b2d5-8fc34f2b3b80 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:53.921174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2fc8b-b0a7-4358-b2d5-8fc34f2b3b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:53.972640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-59a2fc8b-b0a7-4358-b2d5-8fc34f2b3b80 None None] MFA Rules not processed for user `3c2857e783314386960a9970f4854f75`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:53.973287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-59a2fc8b-b0a7-4358-b2d5-8fc34f2b3b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:54.006111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 880/1748] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:54.006111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28756a2c-c22d-45ad-bf86-8fca3d1ac184 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:54.010434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-28756a2c-c22d-45ad-bf86-8fca3d1ac184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:54.049524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 869/1749] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:54.049524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2fc8b-b0a7-4358-b2d5-8fc34f2b3b80 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:54.060380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2fc8b-b0a7-4358-b2d5-8fc34f2b3b80 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:54.065317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b85d2c-c140-4262-8227-5f05fd7a1b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:54.066628 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b85d2c-c140-4262-8227-5f05fd7a1b63 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:54.067023 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b85d2c-c140-4262-8227-5f05fd7a1b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:54.067315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b85d2c-c140-4262-8227-5f05fd7a1b63 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:54.068052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b85d2c-c140-4262-8227-5f05fd7a1b63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:51:54.068360 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b85d2c-c140-4262-8227-5f05fd7a1b63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:54.070167 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b85d2c-c140-4262-8227-5f05fd7a1b63 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:54.084508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 870/1750] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:51:54.085181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b85d2c-c140-4262-8227-5f05fd7a1b63 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:54.096165 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b85d2c-c140-4262-8227-5f05fd7a1b63 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:54.101119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe29c1-f9b8-4e21-8e47-e119457f06be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:54.101793 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe29c1-f9b8-4e21-8e47-e119457f06be admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:54.102078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe29c1-f9b8-4e21-8e47-e119457f06be admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:54.102347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe29c1-f9b8-4e21-8e47-e119457f06be admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:54.103735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe29c1-f9b8-4e21-8e47-e119457f06be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:54.105719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe29c1-f9b8-4e21-8e47-e119457f06be admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:54.109949 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28756a2c-c22d-45ad-bf86-8fca3d1ac184 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:54.118365 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81937f60-d354-4d0d-a519-01740728e77f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:54.119114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81937f60-d354-4d0d-a519-01740728e77f service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:54.119401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81937f60-d354-4d0d-a519-01740728e77f service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:54.119680 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81937f60-d354-4d0d-a519-01740728e77f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:54.142551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 871/1751] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:54 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:54.142551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe29c1-f9b8-4e21-8e47-e119457f06be admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:54.185763 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81937f60-d354-4d0d-a519-01740728e77f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:54.191326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81937f60-d354-4d0d-a519-01740728e77f service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:54.203927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 881/1752] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:54.204772 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81937f60-d354-4d0d-a519-01740728e77f service nova] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:54.218846 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81937f60-d354-4d0d-a519-01740728e77f service nova] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:54.299482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe29c1-f9b8-4e21-8e47-e119457f06be admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:54.319265 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c9135-9a9a-4bb9-8793-5ea084758d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:54.320087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9135-9a9a-4bb9-8793-5ea084758d08 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:54.320281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9135-9a9a-4bb9-8793-5ea084758d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:54.320565 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9135-9a9a-4bb9-8793-5ea084758d08 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:54.321260 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c9135-9a9a-4bb9-8793-5ea084758d08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:54.326561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c9135-9a9a-4bb9-8793-5ea084758d08 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:54.332871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3a272-7f10-4a6f-a10f-79db23bcd6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:54.333599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3a272-7f10-4a6f-a10f-79db23bcd6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:54.333918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3a272-7f10-4a6f-a10f-79db23bcd6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:54.334152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3a272-7f10-4a6f-a10f-79db23bcd6b7 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:54.335713 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3a272-7f10-4a6f-a10f-79db23bcd6b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:54.337814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3a272-7f10-4a6f-a10f-79db23bcd6b7 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:54.384588 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 882/1753] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:54 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:54.387251 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 872/1754] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:54 2022] POST /identity/v3/projects => generated 365 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:54.391297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3a272-7f10-4a6f-a10f-79db23bcd6b7 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:54.396130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c9135-9a9a-4bb9-8793-5ea084758d08 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:54.408572 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3a272-7f10-4a6f-a10f-79db23bcd6b7 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:54.494155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c9135-9a9a-4bb9-8793-5ea084758d08 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:54.500259 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693030d8-f4ce-4549-9cc6-e81775565dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:54.501186 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693030d8-f4ce-4549-9cc6-e81775565dac admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:54.501569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693030d8-f4ce-4549-9cc6-e81775565dac admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:54.502092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693030d8-f4ce-4549-9cc6-e81775565dac admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:54.502928 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693030d8-f4ce-4549-9cc6-e81775565dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:54.505000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693030d8-f4ce-4549-9cc6-e81775565dac admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:54.553569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 873/1755] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:54 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:54.553569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb345e5f-2d3e-4700-bacc-2a56cb478b31 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:54.553569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb345e5f-2d3e-4700-bacc-2a56cb478b31 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:54.553569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb345e5f-2d3e-4700-bacc-2a56cb478b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:54.571416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2cd2e-8f10-4fdf-bb50-97e4de2dddef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:54.572877 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2cd2e-8f10-4fdf-bb50-97e4de2dddef admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:54.573991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2cd2e-8f10-4fdf-bb50-97e4de2dddef admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:54.576638 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2cd2e-8f10-4fdf-bb50-97e4de2dddef admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:54.579025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2cd2e-8f10-4fdf-bb50-97e4de2dddef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:54.583591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2cd2e-8f10-4fdf-bb50-97e4de2dddef admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:54.588955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-eb345e5f-2d3e-4700-bacc-2a56cb478b31 None None] MFA Rules not processed for user `6b11a3de8b71482594a04ab7e53a94bd`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:54.591220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-eb345e5f-2d3e-4700-bacc-2a56cb478b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:54.613576 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 883/1756] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:54.619270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2cd2e-8f10-4fdf-bb50-97e4de2dddef admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:54.639825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2cd2e-8f10-4fdf-bb50-97e4de2dddef admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:54.649900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472bf540-d484-4762-bc27-b4138c491eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:54.650585 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472bf540-d484-4762-bc27-b4138c491eee admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:54.650896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472bf540-d484-4762-bc27-b4138c491eee admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:54.651612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472bf540-d484-4762-bc27-b4138c491eee admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:54.652345 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472bf540-d484-4762-bc27-b4138c491eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:54.653988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472bf540-d484-4762-bc27-b4138c491eee admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:54.664673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 884/1757] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:54.665750 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472bf540-d484-4762-bc27-b4138c491eee admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:54.677188 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472bf540-d484-4762-bc27-b4138c491eee admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:54.681986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc50abe-57ff-4765-96ae-dea99bdae0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:54.682907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc50abe-57ff-4765-96ae-dea99bdae0d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:54.683191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc50abe-57ff-4765-96ae-dea99bdae0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:54.683470 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc50abe-57ff-4765-96ae-dea99bdae0d3 admin admin] PATH_INFO: `/v3/projects/d1d4fbc895b241abb9d8a895756fe9cf/users/89db057e13a346769ca9cdf54acd31c6/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:54.709264 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc50abe-57ff-4765-96ae-dea99bdae0d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d4fbc895b241abb9d8a895756fe9cf, user_id=89db057e13a346769ca9cdf54acd31c6, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:54.713695 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc50abe-57ff-4765-96ae-dea99bdae0d3 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:54.716885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 874/1758] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:54.719897 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb345e5f-2d3e-4700-bacc-2a56cb478b31 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:54.740616 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 885/1759] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:54 2022] PUT /identity/v3/projects/d1d4fbc895b241abb9d8a895756fe9cf/users/89db057e13a346769ca9cdf54acd31c6/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:54.744011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ab955-46da-4c99-a6d1-7b45ceb5e146 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:54.744011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ab955-46da-4c99-a6d1-7b45ceb5e146 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:54.744011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ab955-46da-4c99-a6d1-7b45ceb5e146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:54.745515 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb345e5f-2d3e-4700-bacc-2a56cb478b31 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:54.788105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-069ab955-46da-4c99-a6d1-7b45ceb5e146 None None] MFA Rules not processed for user `6b11a3de8b71482594a04ab7e53a94bd`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:54.790011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-069ab955-46da-4c99-a6d1-7b45ceb5e146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:54.854190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a1255-088a-4809-b65a-4e05882d5988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:54.856661 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a1255-088a-4809-b65a-4e05882d5988 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:54.856903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a1255-088a-4809-b65a-4e05882d5988 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:54.857179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a1255-088a-4809-b65a-4e05882d5988 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:54.860681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a1255-088a-4809-b65a-4e05882d5988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:54.864221 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a1255-088a-4809-b65a-4e05882d5988 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:54.886668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 875/1760] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:54.891517 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a1255-088a-4809-b65a-4e05882d5988 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:54.900841 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 886/1761] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:54.912964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ab955-46da-4c99-a6d1-7b45ceb5e146 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:54.924938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a1255-088a-4809-b65a-4e05882d5988 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:54.932843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ab955-46da-4c99-a6d1-7b45ceb5e146 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:54.936657 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8aee61-8721-4dcd-b3f8-77e596e3fc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:54.937592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8aee61-8721-4dcd-b3f8-77e596e3fc42 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:54.937783 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8aee61-8721-4dcd-b3f8-77e596e3fc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:54.938450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8aee61-8721-4dcd-b3f8-77e596e3fc42 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:54.939791 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8aee61-8721-4dcd-b3f8-77e596e3fc42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:54.942753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8aee61-8721-4dcd-b3f8-77e596e3fc42 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:54.947300 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09ed06-45ce-4e70-91a5-dc56daec5ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:54.948210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09ed06-45ce-4e70-91a5-dc56daec5ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:54.948491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09ed06-45ce-4e70-91a5-dc56daec5ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:54.948767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09ed06-45ce-4e70-91a5-dc56daec5ed4 admin admin] PATH_INFO: `/v3/projects/a8d86efe63d64e8aa2f3aacba8811e24/users/ac76514829f54a8499962c2d6f7bb168/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:54.964983 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 876/1762] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:54.967950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8aee61-8721-4dcd-b3f8-77e596e3fc42 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:54.989317 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09ed06-45ce-4e70-91a5-dc56daec5ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d86efe63d64e8aa2f3aacba8811e24, user_id=ac76514829f54a8499962c2d6f7bb168, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:55.000254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09ed06-45ce-4e70-91a5-dc56daec5ed4 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:55.030133 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 887/1763] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:54 2022] PUT /identity/v3/projects/a8d86efe63d64e8aa2f3aacba8811e24/users/ac76514829f54a8499962c2d6f7bb168/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:55.032538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09ed06-45ce-4e70-91a5-dc56daec5ed4 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:55.135730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8aee61-8721-4dcd-b3f8-77e596e3fc42 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:55.289011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09ed06-45ce-4e70-91a5-dc56daec5ed4 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:55.315959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82b754-92f0-4ed2-9c98-efe59aa0cd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:55.317540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82b754-92f0-4ed2-9c98-efe59aa0cd31 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:55.317850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82b754-92f0-4ed2-9c98-efe59aa0cd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:55.318547 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82b754-92f0-4ed2-9c98-efe59aa0cd31 admin admin] PATH_INFO: `/v3/projects/d1d4fbc895b241abb9d8a895756fe9cf/users/89db057e13a346769ca9cdf54acd31c6/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:55.322418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82b754-92f0-4ed2-9c98-efe59aa0cd31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d4fbc895b241abb9d8a895756fe9cf, user_id=89db057e13a346769ca9cdf54acd31c6, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:55.326612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82b754-92f0-4ed2-9c98-efe59aa0cd31 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:55.343905 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea3b77-7bec-478e-928b-c75b1b2c859d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:55.344992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea3b77-7bec-478e-928b-c75b1b2c859d service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:55.346978 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea3b77-7bec-478e-928b-c75b1b2c859d service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:55.348285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea3b77-7bec-478e-928b-c75b1b2c859d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:55.371997 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 888/1764] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:55 2022] PUT /identity/v3/projects/d1d4fbc895b241abb9d8a895756fe9cf/users/89db057e13a346769ca9cdf54acd31c6/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:55.373729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534235fd-de96-452f-bc88-2516973dbf74 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:55.374022 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534235fd-de96-452f-bc88-2516973dbf74 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:55.374305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534235fd-de96-452f-bc88-2516973dbf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:55.461381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-534235fd-de96-452f-bc88-2516973dbf74 None None] MFA Rules not processed for user `ac76514829f54a8499962c2d6f7bb168`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:55.464357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-534235fd-de96-452f-bc88-2516973dbf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:55.505222 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea3b77-7bec-478e-928b-c75b1b2c859d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:55.510005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea3b77-7bec-478e-928b-c75b1b2c859d service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:55.540082 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 877/1765] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:55.543004 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09dea3-264f-4f41-975c-5c4c7fe0bcc1 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:55.543245 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09dea3-264f-4f41-975c-5c4c7fe0bcc1 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:55.543561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09dea3-264f-4f41-975c-5c4c7fe0bcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:55.614449 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5c09dea3-264f-4f41-975c-5c4c7fe0bcc1 None None] MFA Rules not processed for user `89db057e13a346769ca9cdf54acd31c6`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:55.616754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5c09dea3-264f-4f41-975c-5c4c7fe0bcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:55.751147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 889/1766] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:55.753374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534235fd-de96-452f-bc88-2516973dbf74 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:55.828885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 878/1767] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:55.837428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09dea3-264f-4f41-975c-5c4c7fe0bcc1 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:55.838883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5c09dea3-264f-4f41-975c-5c4c7fe0bcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:55.987883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09dea3-264f-4f41-975c-5c4c7fe0bcc1 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:55.994829 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbac21-6f74-4a24-b7f3-2f6ba05781a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:56.001333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbac21-6f74-4a24-b7f3-2f6ba05781a1 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:56.001706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbac21-6f74-4a24-b7f3-2f6ba05781a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:56.001994 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbac21-6f74-4a24-b7f3-2f6ba05781a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:56.022889 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbac21-6f74-4a24-b7f3-2f6ba05781a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:56.026789 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbac21-6f74-4a24-b7f3-2f6ba05781a1 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:56.049886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534235fd-de96-452f-bc88-2516973dbf74 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:56.062436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 879/1768] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:51:56.083035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a04701-8add-4fdd-b2b8-3fb32d39e76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:56.084239 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a04701-8add-4fdd-b2b8-3fb32d39e76e admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:56.087146 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a04701-8add-4fdd-b2b8-3fb32d39e76e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:56.087627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a04701-8add-4fdd-b2b8-3fb32d39e76e admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:56.089828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a04701-8add-4fdd-b2b8-3fb32d39e76e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:56.101920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a04701-8add-4fdd-b2b8-3fb32d39e76e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:56.190663 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbac21-6f74-4a24-b7f3-2f6ba05781a1 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:56.196526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 890/1769] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:51:55 2022] POST /identity/v3/projects => generated 363 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:56.215308 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbac21-6f74-4a24-b7f3-2f6ba05781a1 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:56.268313 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a04701-8add-4fdd-b2b8-3fb32d39e76e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:56.333267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb19c67-79ff-4ac6-8a16-a60675821678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:56.333267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb19c67-79ff-4ac6-8a16-a60675821678 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:56.333506 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb19c67-79ff-4ac6-8a16-a60675821678 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:56.333837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb19c67-79ff-4ac6-8a16-a60675821678 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:56.334998 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb19c67-79ff-4ac6-8a16-a60675821678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:56.337947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb19c67-79ff-4ac6-8a16-a60675821678 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:56.379677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 880/1770] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:51:56 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:51:56.380862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb19c67-79ff-4ac6-8a16-a60675821678 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:56.396984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a04701-8add-4fdd-b2b8-3fb32d39e76e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:56.401949 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfc01b-f10c-4482-b121-738a1799f0ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:56.403779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfc01b-f10c-4482-b121-738a1799f0ef service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:56.404608 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfc01b-f10c-4482-b121-738a1799f0ef service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:56.406165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfc01b-f10c-4482-b121-738a1799f0ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:56.412854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb19c67-79ff-4ac6-8a16-a60675821678 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:56.419752 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ee877-eaec-4791-9002-52652dcd903f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:56.420861 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ee877-eaec-4791-9002-52652dcd903f admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:56.421204 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ee877-eaec-4791-9002-52652dcd903f admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:56.421560 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ee877-eaec-4791-9002-52652dcd903f admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:56.422384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ee877-eaec-4791-9002-52652dcd903f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:56.424576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ee877-eaec-4791-9002-52652dcd903f admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:56.453802 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 881/1771] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:56.454804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ee877-eaec-4791-9002-52652dcd903f admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:56.484486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ee877-eaec-4791-9002-52652dcd903f admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:56.496223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe1dcd-50f4-4d32-a5d5-a0d3c3865d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:56.496595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe1dcd-50f4-4d32-a5d5-a0d3c3865d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:56.501344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe1dcd-50f4-4d32-a5d5-a0d3c3865d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:56.501344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe1dcd-50f4-4d32-a5d5-a0d3c3865d65 admin admin] PATH_INFO: `/v3/projects/7cb4ccdcc6ef4f19a695dce126085e94/users/09d489b53de9435c8947715194cf7b10/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:56.554200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe1dcd-50f4-4d32-a5d5-a0d3c3865d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb4ccdcc6ef4f19a695dce126085e94, user_id=09d489b53de9435c8947715194cf7b10, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:56.568399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe1dcd-50f4-4d32-a5d5-a0d3c3865d65 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:56.608519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfc01b-f10c-4482-b121-738a1799f0ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:56.610191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfc01b-f10c-4482-b121-738a1799f0ef service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:56.619758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 882/1772] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:56 2022] PUT /identity/v3/projects/7cb4ccdcc6ef4f19a695dce126085e94/users/09d489b53de9435c8947715194cf7b10/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:56.620935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe1dcd-50f4-4d32-a5d5-a0d3c3865d65 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:56.675514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 891/1773] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:51:56.948730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe1dcd-50f4-4d32-a5d5-a0d3c3865d65 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:56.981343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca96ee-b271-4d5c-90e6-6ce46f848b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:56.991032 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca96ee-b271-4d5c-90e6-6ce46f848b4d admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:56.992624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca96ee-b271-4d5c-90e6-6ce46f848b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:56.994291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca96ee-b271-4d5c-90e6-6ce46f848b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:56.995430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca96ee-b271-4d5c-90e6-6ce46f848b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:57.000022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca96ee-b271-4d5c-90e6-6ce46f848b4d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:57.037186 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 883/1774] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:51:57.038430 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfc01b-f10c-4482-b121-738a1799f0ef service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:57.056788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfc01b-f10c-4482-b121-738a1799f0ef service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:57.079258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4c252-ae99-4941-9773-4e659ff08ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:57.081267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4c252-ae99-4941-9773-4e659ff08ebb admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:57.082595 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4c252-ae99-4941-9773-4e659ff08ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:57.082926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4c252-ae99-4941-9773-4e659ff08ebb admin admin] PATH_INFO: `/v3/projects/7cb4ccdcc6ef4f19a695dce126085e94/users/09d489b53de9435c8947715194cf7b10/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:57.089253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4c252-ae99-4941-9773-4e659ff08ebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb4ccdcc6ef4f19a695dce126085e94, user_id=09d489b53de9435c8947715194cf7b10, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:57.097288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4c252-ae99-4941-9773-4e659ff08ebb admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:57.132649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 892/1775] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:51:57 2022] PUT /identity/v3/projects/7cb4ccdcc6ef4f19a695dce126085e94/users/09d489b53de9435c8947715194cf7b10/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:51:57.146590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f44b4-0054-4005-aec4-564ab15a7619 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:57.147261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f44b4-0054-4005-aec4-564ab15a7619 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:57.147800 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f44b4-0054-4005-aec4-564ab15a7619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:57.274093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584d276-0c90-43bb-a482-9b451cd88c28 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:57.274539 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584d276-0c90-43bb-a482-9b451cd88c28 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:57.274943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584d276-0c90-43bb-a482-9b451cd88c28 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:57.279282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 893/1776] 10.222.0.35 () {54 vars in 912 bytes} [Fri May 13 21:51:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:51:57.306605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e770d5c-f6cf-482b-930e-eed2e88a39ea None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:57.307055 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e770d5c-f6cf-482b-930e-eed2e88a39ea None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:57.308642 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e770d5c-f6cf-482b-930e-eed2e88a39ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:57.320092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-2c2f44b4-0054-4005-aec4-564ab15a7619 None None] MFA Rules not processed for user `09d489b53de9435c8947715194cf7b10`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:57.337711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2c2f44b4-0054-4005-aec4-564ab15a7619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:57.392935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-9e770d5c-f6cf-482b-930e-eed2e88a39ea None None] MFA Rules not processed for user `c4df058606c448a1bb757020e869ac33`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:57.403422 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9e770d5c-f6cf-482b-930e-eed2e88a39ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:57.589601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 894/1777] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:51:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:57.602539 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 884/1778] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:57 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:51:57.603033 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e770d5c-f6cf-482b-930e-eed2e88a39ea None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:57.605013 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9e770d5c-f6cf-482b-930e-eed2e88a39ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:57.633859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e770d5c-f6cf-482b-930e-eed2e88a39ea None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:57.646423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3c60d-353f-41c1-9205-aa5f02b6f60a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:57.647131 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3c60d-353f-41c1-9205-aa5f02b6f60a service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:57.647417 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3c60d-353f-41c1-9205-aa5f02b6f60a service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:57.647695 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3c60d-353f-41c1-9205-aa5f02b6f60a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:57.666371 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3c60d-353f-41c1-9205-aa5f02b6f60a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:57.667930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3c60d-353f-41c1-9205-aa5f02b6f60a service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:57.731102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 895/1779] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:51:58.018117 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f44b4-0054-4005-aec4-564ab15a7619 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:58.020287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2c2f44b4-0054-4005-aec4-564ab15a7619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:58.254337 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f44b4-0054-4005-aec4-564ab15a7619 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:58.269283 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aadd1c-82d8-4351-b1dd-715873dc8a20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:58.271661 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aadd1c-82d8-4351-b1dd-715873dc8a20 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:58.275589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aadd1c-82d8-4351-b1dd-715873dc8a20 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:58.275589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aadd1c-82d8-4351-b1dd-715873dc8a20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:58.312765 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aadd1c-82d8-4351-b1dd-715873dc8a20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:58.316553 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aadd1c-82d8-4351-b1dd-715873dc8a20 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:58.381824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 885/1780] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:58 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:58.919891 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3c60d-353f-41c1-9205-aa5f02b6f60a service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:51:58.932314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3c60d-353f-41c1-9205-aa5f02b6f60a service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:51:58.946818 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcd13d-1d8b-4cb9-bc3b-443b08438cd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:51:58.947484 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcd13d-1d8b-4cb9-bc3b-443b08438cd5 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:58.947769 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcd13d-1d8b-4cb9-bc3b-443b08438cd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:58.948047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcd13d-1d8b-4cb9-bc3b-443b08438cd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:59.233005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7151002-a67b-4656-9bf8-52efd315a61f None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:51:59.233595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7151002-a67b-4656-9bf8-52efd315a61f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:51:59.233719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7151002-a67b-4656-9bf8-52efd315a61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:51:59.284739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-a7151002-a67b-4656-9bf8-52efd315a61f None None] MFA Rules not processed for user `6b11a3de8b71482594a04ab7e53a94bd`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:51:59.286106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a7151002-a67b-4656-9bf8-52efd315a61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:51:59.303210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcd13d-1d8b-4cb9-bc3b-443b08438cd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:51:59.304764 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcd13d-1d8b-4cb9-bc3b-443b08438cd5 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:51:59.356621 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 896/1781] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:51:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:51:59.526808 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 886/1782] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:51:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:00.362424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a7e02-a711-4a6d-b64d-ece76bbd69c1 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:00.365771 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a7e02-a711-4a6d-b64d-ece76bbd69c1 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:00.366927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a7e02-a711-4a6d-b64d-ece76bbd69c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:00.406761 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d03a7e02-a711-4a6d-b64d-ece76bbd69c1 None None] MFA Rules not processed for user `0a1425d092844a2e9b8a917dc9981b32`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:00.408408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d03a7e02-a711-4a6d-b64d-ece76bbd69c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:00.581406 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 897/1783] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:03.103442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7151002-a67b-4656-9bf8-52efd315a61f None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:03.163613 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7151002-a67b-4656-9bf8-52efd315a61f None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:03.168083 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c94f50-c630-4001-a42c-5a060d9987fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:03.169556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c94f50-c630-4001-a42c-5a060d9987fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:03.169839 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c94f50-c630-4001-a42c-5a060d9987fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:03.170111 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c94f50-c630-4001-a42c-5a060d9987fa admin admin] PATH_INFO: `/v3/users/ac76514829f54a8499962c2d6f7bb168` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:03.171324 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c94f50-c630-4001-a42c-5a060d9987fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac76514829f54a8499962c2d6f7bb168)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:03.173324 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c94f50-c630-4001-a42c-5a060d9987fa admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:03.217477 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-03c94f50-c630-4001-a42c-5a060d9987fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac76514829f54a8499962c2d6f7bb168'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.229331 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-03c94f50-c630-4001-a42c-5a060d9987fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac76514829f54a8499962c2d6f7bb168'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.230030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-03c94f50-c630-4001-a42c-5a060d9987fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac76514829f54a8499962c2d6f7bb168'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.231275 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-03c94f50-c630-4001-a42c-5a060d9987fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac76514829f54a8499962c2d6f7bb168'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.237971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-03c94f50-c630-4001-a42c-5a060d9987fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac76514829f54a8499962c2d6f7bb168'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.250926 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 887/1784] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:03 2022] DELETE /identity/v3/users/ac76514829f54a8499962c2d6f7bb168 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:03.255446 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5976d9f-99c3-473e-a839-b21318e57e76 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:03.255976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5976d9f-99c3-473e-a839-b21318e57e76 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:03.256065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5976d9f-99c3-473e-a839-b21318e57e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:03.273064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a5976d9f-99c3-473e-a839-b21318e57e76 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:03.274270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a5976d9f-99c3-473e-a839-b21318e57e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:03.335074 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 898/1785] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:03.345635 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c94f50-c630-4001-a42c-5a060d9987fa admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:03.347203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-03c94f50-c630-4001-a42c-5a060d9987fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:03.411873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c94f50-c630-4001-a42c-5a060d9987fa admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:03.419099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599457e5-5f08-43d8-880f-017aba621be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:03.419760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599457e5-5f08-43d8-880f-017aba621be6 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:03.420076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599457e5-5f08-43d8-880f-017aba621be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:03.420383 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599457e5-5f08-43d8-880f-017aba621be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:03.429829 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599457e5-5f08-43d8-880f-017aba621be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:03.431300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599457e5-5f08-43d8-880f-017aba621be6 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:03.440772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 888/1786] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:03.707163 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5976d9f-99c3-473e-a839-b21318e57e76 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:03.708896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a5976d9f-99c3-473e-a839-b21318e57e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:03.738145 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5976d9f-99c3-473e-a839-b21318e57e76 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:03.743756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:03.744570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:03.745038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:03.745469 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] PATH_INFO: `/v3/projects/a8d86efe63d64e8aa2f3aacba8811e24` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:03.747472 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d86efe63d64e8aa2f3aacba8811e24)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:03.749557 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:03.795312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d86efe63d64e8aa2f3aacba8811e24', 'request_id': 'req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.801314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d86efe63d64e8aa2f3aacba8811e24', 'request_id': 'req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.809664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d86efe63d64e8aa2f3aacba8811e24', 'request_id': 'req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.819713 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] The token cache is being invalidate because project a8d86efe63d64e8aa2f3aacba8811e24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:03.820178 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.820852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.822399 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 899/1787] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:03 2022] DELETE /identity/v3/projects/a8d86efe63d64e8aa2f3aacba8811e24 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:03.826464 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599457e5-5f08-43d8-880f-017aba621be6 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:03.836541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599457e5-5f08-43d8-880f-017aba621be6 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:03.900800 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782265b-e2ee-495b-b73e-90f540fb3fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:03.901898 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782265b-e2ee-495b-b73e-90f540fb3fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:03.902253 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782265b-e2ee-495b-b73e-90f540fb3fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:03.903162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782265b-e2ee-495b-b73e-90f540fb3fc6 admin admin] PATH_INFO: `/v3/users/09d489b53de9435c8947715194cf7b10` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:03.904441 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782265b-e2ee-495b-b73e-90f540fb3fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09d489b53de9435c8947715194cf7b10)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:03.906438 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782265b-e2ee-495b-b73e-90f540fb3fc6 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:03.956986 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2782265b-e2ee-495b-b73e-90f540fb3fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09d489b53de9435c8947715194cf7b10'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.972797 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2782265b-e2ee-495b-b73e-90f540fb3fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09d489b53de9435c8947715194cf7b10'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.973529 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2782265b-e2ee-495b-b73e-90f540fb3fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09d489b53de9435c8947715194cf7b10'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.974125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2782265b-e2ee-495b-b73e-90f540fb3fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09d489b53de9435c8947715194cf7b10'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.981189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2782265b-e2ee-495b-b73e-90f540fb3fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09d489b53de9435c8947715194cf7b10'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:03.993888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 889/1788] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:03 2022] DELETE /identity/v3/users/09d489b53de9435c8947715194cf7b10 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:04.268854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:04.270298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:04.285503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6a9cc-c93d-4d9c-affb-c860d165ef0a admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:04.346632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd5c85-432f-440b-8380-82638a4e806f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:04.347534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd5c85-432f-440b-8380-82638a4e806f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:04.347813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd5c85-432f-440b-8380-82638a4e806f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:04.348088 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd5c85-432f-440b-8380-82638a4e806f admin admin] PATH_INFO: `/v3/projects/7cb4ccdcc6ef4f19a695dce126085e94` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:04.349264 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd5c85-432f-440b-8380-82638a4e806f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cb4ccdcc6ef4f19a695dce126085e94)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:04.351256 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd5c85-432f-440b-8380-82638a4e806f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:04.403191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-8edd5c85-432f-440b-8380-82638a4e806f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cb4ccdcc6ef4f19a695dce126085e94', 'request_id': 'req-8edd5c85-432f-440b-8380-82638a4e806f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:04.409674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-8edd5c85-432f-440b-8380-82638a4e806f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cb4ccdcc6ef4f19a695dce126085e94', 'request_id': 'req-8edd5c85-432f-440b-8380-82638a4e806f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:04.420226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-8edd5c85-432f-440b-8380-82638a4e806f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cb4ccdcc6ef4f19a695dce126085e94', 'request_id': 'req-8edd5c85-432f-440b-8380-82638a4e806f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:04.437435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-8edd5c85-432f-440b-8380-82638a4e806f admin admin] The token cache is being invalidate because project 7cb4ccdcc6ef4f19a695dce126085e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:04.437737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-8edd5c85-432f-440b-8380-82638a4e806f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:04.438833 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-8edd5c85-432f-440b-8380-82638a4e806f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:04.444909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 900/1789] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:04 2022] DELETE /identity/v3/projects/7cb4ccdcc6ef4f19a695dce126085e94 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:04.446461 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33d5c4-60ba-42e4-878a-27a9ad0e9057 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:04.446958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33d5c4-60ba-42e4-878a-27a9ad0e9057 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:04.447069 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33d5c4-60ba-42e4-878a-27a9ad0e9057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:04.472863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5f33d5c4-60ba-42e4-878a-27a9ad0e9057 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:04.474430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5f33d5c4-60ba-42e4-878a-27a9ad0e9057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:04.547320 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 890/1790] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:04.549109 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06679c-4ab5-4a81-8b7d-6b220d258434 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:04.549489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06679c-4ab5-4a81-8b7d-6b220d258434 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:04.549835 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06679c-4ab5-4a81-8b7d-6b220d258434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:04.577156 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-6b06679c-4ab5-4a81-8b7d-6b220d258434 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:04.578575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-6b06679c-4ab5-4a81-8b7d-6b220d258434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:04.586801 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 901/1791] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:04.591456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33d5c4-60ba-42e4-878a-27a9ad0e9057 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:04.600945 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33d5c4-60ba-42e4-878a-27a9ad0e9057 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:04.605567 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2034453-b40c-4ec9-9177-415d65b5842b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:04.606219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2034453-b40c-4ec9-9177-415d65b5842b admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:04.606508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2034453-b40c-4ec9-9177-415d65b5842b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:04.606823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2034453-b40c-4ec9-9177-415d65b5842b admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:04.607552 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2034453-b40c-4ec9-9177-415d65b5842b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:04.607863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2034453-b40c-4ec9-9177-415d65b5842b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:04.609501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2034453-b40c-4ec9-9177-415d65b5842b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:04.622959 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 891/1792] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:04.627250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06679c-4ab5-4a81-8b7d-6b220d258434 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:04.636477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06679c-4ab5-4a81-8b7d-6b220d258434 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:04.641876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6337c-01e8-4833-a2e5-b2dad75dc4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:04.642499 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6337c-01e8-4833-a2e5-b2dad75dc4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:04.642886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6337c-01e8-4833-a2e5-b2dad75dc4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:04.643224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6337c-01e8-4833-a2e5-b2dad75dc4a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:04.644003 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6337c-01e8-4833-a2e5-b2dad75dc4a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:04.646003 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6337c-01e8-4833-a2e5-b2dad75dc4a2 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:04.671106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 902/1793] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:04 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:04.675175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2034453-b40c-4ec9-9177-415d65b5842b admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:04.685879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2034453-b40c-4ec9-9177-415d65b5842b admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:04.740309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c8804-0c6b-4c34-b8d3-e20277986014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:04.741008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c8804-0c6b-4c34-b8d3-e20277986014 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:04.741521 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c8804-0c6b-4c34-b8d3-e20277986014 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:04.741800 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c8804-0c6b-4c34-b8d3-e20277986014 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:04.742490 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c8804-0c6b-4c34-b8d3-e20277986014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:04.744530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c8804-0c6b-4c34-b8d3-e20277986014 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:04.764943 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 892/1794] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:04 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:04.768850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6337c-01e8-4833-a2e5-b2dad75dc4a2 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:04.779059 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6337c-01e8-4833-a2e5-b2dad75dc4a2 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:04.783641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c57712-1466-47de-bceb-f91e11cc0e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:04.784289 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c57712-1466-47de-bceb-f91e11cc0e11 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:04.784573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c57712-1466-47de-bceb-f91e11cc0e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:04.784848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c57712-1466-47de-bceb-f91e11cc0e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:04.785562 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c57712-1466-47de-bceb-f91e11cc0e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:04.787188 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c57712-1466-47de-bceb-f91e11cc0e11 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:04.797200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 903/1795] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:04.800845 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c8804-0c6b-4c34-b8d3-e20277986014 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:04.810854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c8804-0c6b-4c34-b8d3-e20277986014 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:04.815408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08434ba0-8170-43fb-96a5-8e168330f1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:04.816043 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08434ba0-8170-43fb-96a5-8e168330f1af admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:04.816323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08434ba0-8170-43fb-96a5-8e168330f1af admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:04.816593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08434ba0-8170-43fb-96a5-8e168330f1af admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:04.817298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08434ba0-8170-43fb-96a5-8e168330f1af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:04.818897 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08434ba0-8170-43fb-96a5-8e168330f1af admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:04.830003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 893/1796] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:04.833773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c57712-1466-47de-bceb-f91e11cc0e11 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:04.843513 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c57712-1466-47de-bceb-f91e11cc0e11 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:04.848093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441267f4-93c0-4395-9791-74c033186cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:04.848945 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441267f4-93c0-4395-9791-74c033186cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:04.849228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441267f4-93c0-4395-9791-74c033186cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:04.849503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441267f4-93c0-4395-9791-74c033186cf7 admin admin] PATH_INFO: `/v3/projects/f189a77f1be940449545b2df5d27690a/users/12367add319647d88f447db8bd31ab2e/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:04.863362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441267f4-93c0-4395-9791-74c033186cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f189a77f1be940449545b2df5d27690a, user_id=12367add319647d88f447db8bd31ab2e, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:04.866010 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441267f4-93c0-4395-9791-74c033186cf7 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:04.880365 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 904/1797] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:04 2022] PUT /identity/v3/projects/f189a77f1be940449545b2df5d27690a/users/12367add319647d88f447db8bd31ab2e/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:04.894352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efceca-d6c6-455e-b56b-af4f5978d819 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:04.894639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efceca-d6c6-455e-b56b-af4f5978d819 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:04.894936 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efceca-d6c6-455e-b56b-af4f5978d819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:04.920777 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-90efceca-d6c6-455e-b56b-af4f5978d819 None None] MFA Rules not processed for user `12367add319647d88f447db8bd31ab2e`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:04.921968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-90efceca-d6c6-455e-b56b-af4f5978d819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:04.987415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 894/1798] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:04.988649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441267f4-93c0-4395-9791-74c033186cf7 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:05.047852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441267f4-93c0-4395-9791-74c033186cf7 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:05.052475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4fa240-9adc-4422-ace8-f5773a0b6c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:05.053121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fa240-9adc-4422-ace8-f5773a0b6c3c admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:05.053404 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fa240-9adc-4422-ace8-f5773a0b6c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:05.053678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fa240-9adc-4422-ace8-f5773a0b6c3c admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:05.054356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4fa240-9adc-4422-ace8-f5773a0b6c3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:05.056285 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4fa240-9adc-4422-ace8-f5773a0b6c3c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:05.081172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 905/1799] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:04 2022] POST /identity/v3/projects => generated 369 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:05.085309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efceca-d6c6-455e-b56b-af4f5978d819 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:05.096184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efceca-d6c6-455e-b56b-af4f5978d819 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:05.165846 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cefd195-b4a6-477b-a6d7-c228ec700d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:05.166529 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cefd195-b4a6-477b-a6d7-c228ec700d1c admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:05.166852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cefd195-b4a6-477b-a6d7-c228ec700d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:05.167135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cefd195-b4a6-477b-a6d7-c228ec700d1c admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:05.167844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cefd195-b4a6-477b-a6d7-c228ec700d1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:05.170446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cefd195-b4a6-477b-a6d7-c228ec700d1c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:05.193338 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 895/1800] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:05 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:05.194669 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4fa240-9adc-4422-ace8-f5773a0b6c3c admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:05.204878 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4fa240-9adc-4422-ace8-f5773a0b6c3c admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:05.209702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535a643-15e6-45f5-acda-e609b8fe48db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:05.210349 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535a643-15e6-45f5-acda-e609b8fe48db admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:05.210630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535a643-15e6-45f5-acda-e609b8fe48db admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:05.210936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535a643-15e6-45f5-acda-e609b8fe48db admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:05.211649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535a643-15e6-45f5-acda-e609b8fe48db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:05.213242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535a643-15e6-45f5-acda-e609b8fe48db admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:05.223467 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 906/1801] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:05.227316 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cefd195-b4a6-477b-a6d7-c228ec700d1c admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:05.237764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cefd195-b4a6-477b-a6d7-c228ec700d1c admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:05.242496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157119c8-823a-4f9a-bdd5-9b1c2cd644dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:05.243403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157119c8-823a-4f9a-bdd5-9b1c2cd644dc admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:05.243704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157119c8-823a-4f9a-bdd5-9b1c2cd644dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:05.243984 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157119c8-823a-4f9a-bdd5-9b1c2cd644dc admin admin] PATH_INFO: `/v3/projects/97153c15fd6843308c4c8ad6f853e4b5/users/9467dac9be484461b8e00af9a56d676f/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:05.258819 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157119c8-823a-4f9a-bdd5-9b1c2cd644dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=97153c15fd6843308c4c8ad6f853e4b5, user_id=9467dac9be484461b8e00af9a56d676f, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:05.261473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157119c8-823a-4f9a-bdd5-9b1c2cd644dc admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:05.273953 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 896/1802] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:05 2022] PUT /identity/v3/projects/97153c15fd6843308c4c8ad6f853e4b5/users/9467dac9be484461b8e00af9a56d676f/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:05.277723 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535a643-15e6-45f5-acda-e609b8fe48db admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:05.287343 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535a643-15e6-45f5-acda-e609b8fe48db admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:05.341669 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c3a8e-c69b-480e-8da1-25a422863571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:05.342322 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c3a8e-c69b-480e-8da1-25a422863571 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:05.342602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c3a8e-c69b-480e-8da1-25a422863571 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:05.343466 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c3a8e-c69b-480e-8da1-25a422863571 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:05.344185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c3a8e-c69b-480e-8da1-25a422863571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:05.345774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c3a8e-c69b-480e-8da1-25a422863571 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:05.355222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 907/1803] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:05.360082 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157119c8-823a-4f9a-bdd5-9b1c2cd644dc admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:05.370570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157119c8-823a-4f9a-bdd5-9b1c2cd644dc admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:05.375077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6316f-f049-4445-8886-51e137f50153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:05.375940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6316f-f049-4445-8886-51e137f50153 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:05.376220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6316f-f049-4445-8886-51e137f50153 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:05.376493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6316f-f049-4445-8886-51e137f50153 admin admin] PATH_INFO: `/v3/projects/97153c15fd6843308c4c8ad6f853e4b5/users/9467dac9be484461b8e00af9a56d676f/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:05.378413 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6316f-f049-4445-8886-51e137f50153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97153c15fd6843308c4c8ad6f853e4b5, user_id=9467dac9be484461b8e00af9a56d676f, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:05.381065 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6316f-f049-4445-8886-51e137f50153 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:05.390877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 897/1804] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:05 2022] PUT /identity/v3/projects/97153c15fd6843308c4c8ad6f853e4b5/users/9467dac9be484461b8e00af9a56d676f/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:05.401520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e00873-3fa0-4451-a1f8-b026f3505e20 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:05.401809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e00873-3fa0-4451-a1f8-b026f3505e20 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:05.402091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e00873-3fa0-4451-a1f8-b026f3505e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:05.427937 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-77e00873-3fa0-4451-a1f8-b026f3505e20 None None] MFA Rules not processed for user `9467dac9be484461b8e00af9a56d676f`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:05.429111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-77e00873-3fa0-4451-a1f8-b026f3505e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:05.495266 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 908/1805] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:05 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:05.505739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6316f-f049-4445-8886-51e137f50153 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:05.507455 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a5f6316f-f049-4445-8886-51e137f50153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:05.562939 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6316f-f049-4445-8886-51e137f50153 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:05.566809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d8268-de3f-47ab-86a1-f774c9a0beb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:05.567429 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d8268-de3f-47ab-86a1-f774c9a0beb4 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:05.567708 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d8268-de3f-47ab-86a1-f774c9a0beb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:05.567979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d8268-de3f-47ab-86a1-f774c9a0beb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:05.577214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d8268-de3f-47ab-86a1-f774c9a0beb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:05.578639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d8268-de3f-47ab-86a1-f774c9a0beb4 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:05.588466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 898/1806] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:05 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:07.864150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e00873-3fa0-4451-a1f8-b026f3505e20 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:07.868562 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-77e00873-3fa0-4451-a1f8-b026f3505e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:08.023009 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e00873-3fa0-4451-a1f8-b026f3505e20 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:08.044695 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef530775-f4a7-4047-b467-ce9236daa3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:08.046571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef530775-f4a7-4047-b467-ce9236daa3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:08.046962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef530775-f4a7-4047-b467-ce9236daa3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:08.047513 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef530775-f4a7-4047-b467-ce9236daa3e6 admin admin] PATH_INFO: `/v3/users/a457fbf9357347349af2d1d8bd8b5ed8` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:08.049122 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef530775-f4a7-4047-b467-ce9236daa3e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a457fbf9357347349af2d1d8bd8b5ed8)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:08.051471 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef530775-f4a7-4047-b467-ce9236daa3e6 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:08.153410 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ef530775-f4a7-4047-b467-ce9236daa3e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a457fbf9357347349af2d1d8bd8b5ed8'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.193656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ef530775-f4a7-4047-b467-ce9236daa3e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a457fbf9357347349af2d1d8bd8b5ed8'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.195584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ef530775-f4a7-4047-b467-ce9236daa3e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a457fbf9357347349af2d1d8bd8b5ed8'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.205357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ef530775-f4a7-4047-b467-ce9236daa3e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a457fbf9357347349af2d1d8bd8b5ed8'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.222218 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ef530775-f4a7-4047-b467-ce9236daa3e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a457fbf9357347349af2d1d8bd8b5ed8'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.228979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d8268-de3f-47ab-86a1-f774c9a0beb4 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:08.232620 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5e4d8268-de3f-47ab-86a1-f774c9a0beb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:08.262996 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d8268-de3f-47ab-86a1-f774c9a0beb4 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:08.267335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 909/1807] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:07 2022] DELETE /identity/v3/users/a457fbf9357347349af2d1d8bd8b5ed8 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:08.273244 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed798885-54ad-4b78-b19c-c2f5f7d38938 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:08.273923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed798885-54ad-4b78-b19c-c2f5f7d38938 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:08.274560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed798885-54ad-4b78-b19c-c2f5f7d38938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:08.323856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-ed798885-54ad-4b78-b19c-c2f5f7d38938 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:08.325088 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ed798885-54ad-4b78-b19c-c2f5f7d38938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:08.391130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3883435-4f32-463a-be0a-61ec8ab2a84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:08.391130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3883435-4f32-463a-be0a-61ec8ab2a84c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:08.391130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3883435-4f32-463a-be0a-61ec8ab2a84c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:08.391130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3883435-4f32-463a-be0a-61ec8ab2a84c admin admin] PATH_INFO: `/v3/users/26ea13fb1bc54878a47c201e9bde9996` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:08.391408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3883435-4f32-463a-be0a-61ec8ab2a84c admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ea13fb1bc54878a47c201e9bde9996)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:08.391408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3883435-4f32-463a-be0a-61ec8ab2a84c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:08.452131 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 910/1808] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:08.453232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed798885-54ad-4b78-b19c-c2f5f7d38938 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:08.454946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ed798885-54ad-4b78-b19c-c2f5f7d38938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:08.470429 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed798885-54ad-4b78-b19c-c2f5f7d38938 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:08.476220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7d77f-e4e8-499e-81b0-1b4411af1847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:08.477334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7d77f-e4e8-499e-81b0-1b4411af1847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:08.477764 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7d77f-e4e8-499e-81b0-1b4411af1847 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:08.478288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7d77f-e4e8-499e-81b0-1b4411af1847 admin admin] PATH_INFO: `/v3/users/12367add319647d88f447db8bd31ab2e` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:08.479750 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7d77f-e4e8-499e-81b0-1b4411af1847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12367add319647d88f447db8bd31ab2e)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:08.482682 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7d77f-e4e8-499e-81b0-1b4411af1847 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:08.490592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c3883435-4f32-463a-be0a-61ec8ab2a84c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ea13fb1bc54878a47c201e9bde9996'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.517086 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c3883435-4f32-463a-be0a-61ec8ab2a84c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ea13fb1bc54878a47c201e9bde9996'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.517934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c3883435-4f32-463a-be0a-61ec8ab2a84c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ea13fb1bc54878a47c201e9bde9996'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.519205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c3883435-4f32-463a-be0a-61ec8ab2a84c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ea13fb1bc54878a47c201e9bde9996'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.534946 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c3883435-4f32-463a-be0a-61ec8ab2a84c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ea13fb1bc54878a47c201e9bde9996'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.557617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 899/1809] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:08 2022] DELETE /identity/v3/users/26ea13fb1bc54878a47c201e9bde9996 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:08.560084 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3883435-4f32-463a-be0a-61ec8ab2a84c admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:08.562271 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c3883435-4f32-463a-be0a-61ec8ab2a84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:08.568643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6cc7d77f-e4e8-499e-81b0-1b4411af1847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12367add319647d88f447db8bd31ab2e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.592290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6cc7d77f-e4e8-499e-81b0-1b4411af1847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12367add319647d88f447db8bd31ab2e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.593041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6cc7d77f-e4e8-499e-81b0-1b4411af1847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12367add319647d88f447db8bd31ab2e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.593675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6cc7d77f-e4e8-499e-81b0-1b4411af1847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12367add319647d88f447db8bd31ab2e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.601506 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6cc7d77f-e4e8-499e-81b0-1b4411af1847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12367add319647d88f447db8bd31ab2e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:08.630391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 911/1810] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:08 2022] DELETE /identity/v3/users/12367add319647d88f447db8bd31ab2e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:08.635886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314c5d9-51cb-44a8-b543-df3e99c91bfd None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:08.636181 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314c5d9-51cb-44a8-b543-df3e99c91bfd None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:08.636462 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314c5d9-51cb-44a8-b543-df3e99c91bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:08.672800 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3883435-4f32-463a-be0a-61ec8ab2a84c admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:08.674332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-b314c5d9-51cb-44a8-b543-df3e99c91bfd None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:08.675601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b314c5d9-51cb-44a8-b543-df3e99c91bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:08.755552 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea21cf-a25c-4bfe-b519-d967aaed5dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:08.756388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea21cf-a25c-4bfe-b519-d967aaed5dab service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:08.756938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea21cf-a25c-4bfe-b519-d967aaed5dab service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:08.757439 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea21cf-a25c-4bfe-b519-d967aaed5dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:08.759963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-adea21cf-a25c-4bfe-b519-d967aaed5dab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:08.852565 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea21cf-a25c-4bfe-b519-d967aaed5dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:08.870478 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea21cf-a25c-4bfe-b519-d967aaed5dab service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:08.892620 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 900/1811] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:08.894427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e90449-3312-4573-aaa6-9f53fec23553 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:08.895688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e90449-3312-4573-aaa6-9f53fec23553 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:08.897256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e90449-3312-4573-aaa6-9f53fec23553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:08.923267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-36e90449-3312-4573-aaa6-9f53fec23553 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:08.925240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-36e90449-3312-4573-aaa6-9f53fec23553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:08.937417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 901/1812] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:08.958497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e90449-3312-4573-aaa6-9f53fec23553 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:08.962448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-36e90449-3312-4573-aaa6-9f53fec23553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:08.999105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e90449-3312-4573-aaa6-9f53fec23553 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:09.012204 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308da032-7137-45af-9587-d032bec5a1cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:09.013588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308da032-7137-45af-9587-d032bec5a1cd service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:09.014024 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308da032-7137-45af-9587-d032bec5a1cd service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:09.014432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308da032-7137-45af-9587-d032bec5a1cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:09.031917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 912/1813] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:09.038344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308da032-7137-45af-9587-d032bec5a1cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:09.039857 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308da032-7137-45af-9587-d032bec5a1cd service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:09.050955 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314c5d9-51cb-44a8-b543-df3e99c91bfd None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:09.091697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 902/1814] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:09.092269 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314c5d9-51cb-44a8-b543-df3e99c91bfd None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:09.111789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c91771-fa08-4a69-8186-fbe7d42e109b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:09.112911 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c91771-fa08-4a69-8186-fbe7d42e109b service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:09.113604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c91771-fa08-4a69-8186-fbe7d42e109b service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:09.114015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c91771-fa08-4a69-8186-fbe7d42e109b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:09.136348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c715-0b45-4c0b-bcd8-69dafd6c5f7a None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:09.136644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c715-0b45-4c0b-bcd8-69dafd6c5f7a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:09.136926 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c715-0b45-4c0b-bcd8-69dafd6c5f7a None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:09.139344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 903/1815] 10.222.0.35 () {54 vars in 921 bytes} [Fri May 13 21:52:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:52:09.139682 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c91771-fa08-4a69-8186-fbe7d42e109b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:09.141275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c91771-fa08-4a69-8186-fbe7d42e109b service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:09.150826 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59093b01-f71b-4f86-a8f2-8f33103ea4b7 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:09.151436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59093b01-f71b-4f86-a8f2-8f33103ea4b7 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:09.152064 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59093b01-f71b-4f86-a8f2-8f33103ea4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:09.176070 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 913/1816] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:52:09.200255 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-59093b01-f71b-4f86-a8f2-8f33103ea4b7 None None] MFA Rules not processed for user `316ef43c6c304bd3a27e085bf4da8a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:09.203030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-59093b01-f71b-4f86-a8f2-8f33103ea4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:09.391864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 904/1817] 10.222.0.35 () {58 vars in 1018 bytes} [Fri May 13 21:52:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:09.415238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c91771-fa08-4a69-8186-fbe7d42e109b service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:09.418043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-70c91771-fa08-4a69-8186-fbe7d42e109b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:09.453988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c91771-fa08-4a69-8186-fbe7d42e109b service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:09.464435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c424e84-eeff-47cc-8b20-98006fbdac26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:09.465526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c424e84-eeff-47cc-8b20-98006fbdac26 service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:09.466332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c424e84-eeff-47cc-8b20-98006fbdac26 service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:09.468251 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c424e84-eeff-47cc-8b20-98006fbdac26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:09.480312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c424e84-eeff-47cc-8b20-98006fbdac26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:09.483451 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c424e84-eeff-47cc-8b20-98006fbdac26 service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:09.496268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 914/1818] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:52:10.238192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59093b01-f71b-4f86-a8f2-8f33103ea4b7 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:10.239700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-59093b01-f71b-4f86-a8f2-8f33103ea4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:10.253697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59093b01-f71b-4f86-a8f2-8f33103ea4b7 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:10.287109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:10.289119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:10.289640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:10.290147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] PATH_INFO: `/v3/projects/48cfbf08a65a4814bb083dd255d0dd13` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:10.295280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48cfbf08a65a4814bb083dd255d0dd13)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:10.303609 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:10.378558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48cfbf08a65a4814bb083dd255d0dd13', 'request_id': 'req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.387579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48cfbf08a65a4814bb083dd255d0dd13', 'request_id': 'req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.400293 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48cfbf08a65a4814bb083dd255d0dd13', 'request_id': 'req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.427112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] The token cache is being invalidate because project 48cfbf08a65a4814bb083dd255d0dd13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:10.427551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.428381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.430150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 905/1819] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:10 2022] DELETE /identity/v3/projects/48cfbf08a65a4814bb083dd255d0dd13 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:10.434158 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c424e84-eeff-47cc-8b20-98006fbdac26 service nova] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:10.459610 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:10.463291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:10.469054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c424e84-eeff-47cc-8b20-98006fbdac26 service nova] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:10.652985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cffc7-b1b1-46b4-b55d-dda293b6c201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:10.653668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1a9ec-22e7-4eed-bc28-49f9558910f0 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:10.654483 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cffc7-b1b1-46b4-b55d-dda293b6c201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:10.654939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cffc7-b1b1-46b4-b55d-dda293b6c201 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:10.655373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cffc7-b1b1-46b4-b55d-dda293b6c201 admin admin] PATH_INFO: `/v3/users/074ab3f373da4cdfbeb33d19323b2012` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:10.659588 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cffc7-b1b1-46b4-b55d-dda293b6c201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=074ab3f373da4cdfbeb33d19323b2012)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:10.666601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:10.668573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:10.669140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:10.671101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] PATH_INFO: `/v3/projects/f189a77f1be940449545b2df5d27690a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:10.675540 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f189a77f1be940449545b2df5d27690a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:10.677642 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:10.685822 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cffc7-b1b1-46b4-b55d-dda293b6c201 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:10.769873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f189a77f1be940449545b2df5d27690a', 'request_id': 'req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.774556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3c8cffc7-b1b1-46b4-b55d-dda293b6c201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '074ab3f373da4cdfbeb33d19323b2012'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.778904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f189a77f1be940449545b2df5d27690a', 'request_id': 'req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.790396 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f189a77f1be940449545b2df5d27690a', 'request_id': 'req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.799053 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3c8cffc7-b1b1-46b4-b55d-dda293b6c201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '074ab3f373da4cdfbeb33d19323b2012'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.807231 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3c8cffc7-b1b1-46b4-b55d-dda293b6c201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '074ab3f373da4cdfbeb33d19323b2012'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.810975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] The token cache is being invalidate because project f189a77f1be940449545b2df5d27690a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:10.811337 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.812033 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.814006 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3c8cffc7-b1b1-46b4-b55d-dda293b6c201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '074ab3f373da4cdfbeb33d19323b2012'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.818058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 906/1820] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:10 2022] DELETE /identity/v3/projects/f189a77f1be940449545b2df5d27690a => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:10.818058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:10.822812 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:10.830767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3c8cffc7-b1b1-46b4-b55d-dda293b6c201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '074ab3f373da4cdfbeb33d19323b2012'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:10.858226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 915/1821] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:10 2022] DELETE /identity/v3/users/074ab3f373da4cdfbeb33d19323b2012 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:10.860102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cffc7-b1b1-46b4-b55d-dda293b6c201 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:10.863826 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3c8cffc7-b1b1-46b4-b55d-dda293b6c201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:10.929400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672e58b-f80b-4b55-8e6d-a49b2463a9c4 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:11.019317 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cffc7-b1b1-46b4-b55d-dda293b6c201 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:11.029781 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1b4d2-3ffb-4be2-8bd8-bda11bfe6cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:11.030683 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1b4d2-3ffb-4be2-8bd8-bda11bfe6cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:11.031399 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1b4d2-3ffb-4be2-8bd8-bda11bfe6cce admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:11.031399 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1b4d2-3ffb-4be2-8bd8-bda11bfe6cce admin admin] PATH_INFO: `/v3/users/9467dac9be484461b8e00af9a56d676f` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:11.034215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1b4d2-3ffb-4be2-8bd8-bda11bfe6cce admin admin] RBAC: Authorizing `identity:delete_user(user_id=9467dac9be484461b8e00af9a56d676f)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:11.036527 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1b4d2-3ffb-4be2-8bd8-bda11bfe6cce admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:11.074652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:11.076851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:11.076851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:11.077077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8 admin admin] PATH_INFO: `/v3/projects/79e233fe1abb4beda8f9d9755c0482f9` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:11.080881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e233fe1abb4beda8f9d9755c0482f9)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:11.085739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:11.157603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-39e1b4d2-3ffb-4be2-8bd8-bda11bfe6cce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9467dac9be484461b8e00af9a56d676f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.197152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-39e1b4d2-3ffb-4be2-8bd8-bda11bfe6cce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9467dac9be484461b8e00af9a56d676f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.198241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-39e1b4d2-3ffb-4be2-8bd8-bda11bfe6cce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9467dac9be484461b8e00af9a56d676f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.199115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-39e1b4d2-3ffb-4be2-8bd8-bda11bfe6cce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9467dac9be484461b8e00af9a56d676f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.210324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-39e1b4d2-3ffb-4be2-8bd8-bda11bfe6cce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9467dac9be484461b8e00af9a56d676f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.211907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e233fe1abb4beda8f9d9755c0482f9', 'request_id': 'req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.238612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 916/1822] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:10 2022] DELETE /identity/v3/users/9467dac9be484461b8e00af9a56d676f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:11.242797 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee0d27-e974-4086-8b5f-5fb6cd1774d2 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:11.243113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee0d27-e974-4086-8b5f-5fb6cd1774d2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:11.243401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee0d27-e974-4086-8b5f-5fb6cd1774d2 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:11.245285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e233fe1abb4beda8f9d9755c0482f9', 'request_id': 'req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.248622 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 917/1823] 10.222.0.35 () {54 vars in 921 bytes} [Fri May 13 21:52:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:52:11.271194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e233fe1abb4beda8f9d9755c0482f9', 'request_id': 'req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.296053 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8 admin admin] The token cache is being invalidate because project 79e233fe1abb4beda8f9d9755c0482f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:11.296660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.297873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f5feb47e-0b4b-49c8-b422-a8e52c2552c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.304093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 907/1824] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:10 2022] DELETE /identity/v3/projects/79e233fe1abb4beda8f9d9755c0482f9 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:11.308171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee0d27-e974-4086-8b5f-5fb6cd1774d2 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:11.313398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-43ee0d27-e974-4086-8b5f-5fb6cd1774d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:11.360147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f945d3b-6c83-443d-a5f9-6b3ca0e2d9a0 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:11.361643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f945d3b-6c83-443d-a5f9-6b3ca0e2d9a0 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:11.361643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f945d3b-6c83-443d-a5f9-6b3ca0e2d9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:11.402869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-9f945d3b-6c83-443d-a5f9-6b3ca0e2d9a0 None None] MFA Rules not processed for user `316ef43c6c304bd3a27e085bf4da8a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:11.407208 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-9f945d3b-6c83-443d-a5f9-6b3ca0e2d9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:11.567969 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee0d27-e974-4086-8b5f-5fb6cd1774d2 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:11.573667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c511e28-b95a-40a2-bea2-344fddf9d2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:11.574515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c511e28-b95a-40a2-bea2-344fddf9d2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:11.574834 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c511e28-b95a-40a2-bea2-344fddf9d2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:11.575365 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c511e28-b95a-40a2-bea2-344fddf9d2d3 admin admin] PATH_INFO: `/v3/users/1544a91ba9d54b859e35a92cfcf3cdb9` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:11.576909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c511e28-b95a-40a2-bea2-344fddf9d2d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1544a91ba9d54b859e35a92cfcf3cdb9)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:11.579254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c511e28-b95a-40a2-bea2-344fddf9d2d3 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:11.596172 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 908/1825] 10.222.0.35 () {58 vars in 1018 bytes} [Fri May 13 21:52:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:11.659415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0c511e28-b95a-40a2-bea2-344fddf9d2d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1544a91ba9d54b859e35a92cfcf3cdb9'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.703719 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0c511e28-b95a-40a2-bea2-344fddf9d2d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1544a91ba9d54b859e35a92cfcf3cdb9'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.705135 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0c511e28-b95a-40a2-bea2-344fddf9d2d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1544a91ba9d54b859e35a92cfcf3cdb9'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.707508 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0c511e28-b95a-40a2-bea2-344fddf9d2d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1544a91ba9d54b859e35a92cfcf3cdb9'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.716811 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f945d3b-6c83-443d-a5f9-6b3ca0e2d9a0 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:11.718236 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-9f945d3b-6c83-443d-a5f9-6b3ca0e2d9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:11.718754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0c511e28-b95a-40a2-bea2-344fddf9d2d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1544a91ba9d54b859e35a92cfcf3cdb9'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.732866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f945d3b-6c83-443d-a5f9-6b3ca0e2d9a0 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:11.742657 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a614d8b-9d30-4489-b6de-2e32ea965f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:11.743117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 918/1826] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:11 2022] DELETE /identity/v3/users/1544a91ba9d54b859e35a92cfcf3cdb9 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:11.743655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a614d8b-9d30-4489-b6de-2e32ea965f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:11.743894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a614d8b-9d30-4489-b6de-2e32ea965f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:11.744181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a614d8b-9d30-4489-b6de-2e32ea965f09 admin admin] PATH_INFO: `/v3/users/5c3fcbe716c74090b101a068e1ab2ec7` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:11.746184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a614d8b-9d30-4489-b6de-2e32ea965f09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c3fcbe716c74090b101a068e1ab2ec7)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:11.748196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a614d8b-9d30-4489-b6de-2e32ea965f09 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:11.805749 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6a614d8b-9d30-4489-b6de-2e32ea965f09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c3fcbe716c74090b101a068e1ab2ec7'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.822920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6a614d8b-9d30-4489-b6de-2e32ea965f09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c3fcbe716c74090b101a068e1ab2ec7'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.823932 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6a614d8b-9d30-4489-b6de-2e32ea965f09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c3fcbe716c74090b101a068e1ab2ec7'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.824731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6a614d8b-9d30-4489-b6de-2e32ea965f09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c3fcbe716c74090b101a068e1ab2ec7'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.834609 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6a614d8b-9d30-4489-b6de-2e32ea965f09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c3fcbe716c74090b101a068e1ab2ec7'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:11.852904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 909/1827] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:11 2022] DELETE /identity/v3/users/5c3fcbe716c74090b101a068e1ab2ec7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:11.857455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d248c-0fe2-48d5-b659-6bae0facd9e3 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:11.857811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d248c-0fe2-48d5-b659-6bae0facd9e3 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:11.858166 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d248c-0fe2-48d5-b659-6bae0facd9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:11.876823 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-068d248c-0fe2-48d5-b659-6bae0facd9e3 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:11.878197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-068d248c-0fe2-48d5-b659-6bae0facd9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:11.959005 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 919/1828] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:11.992742 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a614d8b-9d30-4489-b6de-2e32ea965f09 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:11.995161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-6a614d8b-9d30-4489-b6de-2e32ea965f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:12.047648 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d248c-0fe2-48d5-b659-6bae0facd9e3 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:12.050058 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-068d248c-0fe2-48d5-b659-6bae0facd9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:12.097970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a614d8b-9d30-4489-b6de-2e32ea965f09 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:12.110605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8707c8-4567-49de-a142-6a5cfbefa6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:12.111296 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8707c8-4567-49de-a142-6a5cfbefa6ce service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:12.111582 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8707c8-4567-49de-a142-6a5cfbefa6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:12.111860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8707c8-4567-49de-a142-6a5cfbefa6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:12.128907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8707c8-4567-49de-a142-6a5cfbefa6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:12.131327 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8707c8-4567-49de-a142-6a5cfbefa6ce service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:12.149296 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 910/1829] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:12.156084 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8707c8-4567-49de-a142-6a5cfbefa6ce service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:12.158186 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-cb8707c8-4567-49de-a142-6a5cfbefa6ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:12.172457 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8707c8-4567-49de-a142-6a5cfbefa6ce service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:12.179091 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948135f6-7eec-43d9-8162-fbe079280b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:12.180086 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d248c-0fe2-48d5-b659-6bae0facd9e3 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:12.180542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948135f6-7eec-43d9-8162-fbe079280b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:12.181034 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948135f6-7eec-43d9-8162-fbe079280b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:12.181636 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948135f6-7eec-43d9-8162-fbe079280b1f admin admin] PATH_INFO: `/v3/projects/97153c15fd6843308c4c8ad6f853e4b5` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:12.183317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948135f6-7eec-43d9-8162-fbe079280b1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=97153c15fd6843308c4c8ad6f853e4b5)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:12.186179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948135f6-7eec-43d9-8162-fbe079280b1f admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:12.187191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24959df-e3d6-4ae6-b394-1a8b603ea1d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:12.187953 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24959df-e3d6-4ae6-b394-1a8b603ea1d4 service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:12.188361 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24959df-e3d6-4ae6-b394-1a8b603ea1d4 service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:12.188741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24959df-e3d6-4ae6-b394-1a8b603ea1d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:12.190665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c24959df-e3d6-4ae6-b394-1a8b603ea1d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:12.208604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24959df-e3d6-4ae6-b394-1a8b603ea1d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:12.210893 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24959df-e3d6-4ae6-b394-1a8b603ea1d4 service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:12.239151 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 920/1830] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:52:12.261099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-948135f6-7eec-43d9-8162-fbe079280b1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97153c15fd6843308c4c8ad6f853e4b5', 'request_id': 'req-948135f6-7eec-43d9-8162-fbe079280b1f'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:12.275982 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-948135f6-7eec-43d9-8162-fbe079280b1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97153c15fd6843308c4c8ad6f853e4b5', 'request_id': 'req-948135f6-7eec-43d9-8162-fbe079280b1f'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:12.285396 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-948135f6-7eec-43d9-8162-fbe079280b1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97153c15fd6843308c4c8ad6f853e4b5', 'request_id': 'req-948135f6-7eec-43d9-8162-fbe079280b1f'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:12.298619 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-948135f6-7eec-43d9-8162-fbe079280b1f admin admin] The token cache is being invalidate because project 97153c15fd6843308c4c8ad6f853e4b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:12.299027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-948135f6-7eec-43d9-8162-fbe079280b1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:12.300077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-948135f6-7eec-43d9-8162-fbe079280b1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:12.301929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 911/1831] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:12 2022] DELETE /identity/v3/projects/97153c15fd6843308c4c8ad6f853e4b5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:12.305906 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24959df-e3d6-4ae6-b394-1a8b603ea1d4 service nova] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:12.307101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8c73a-42d2-4260-9cf2-5c5d29662e8d None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:12.307404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8c73a-42d2-4260-9cf2-5c5d29662e8d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:12.307566 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c24959df-e3d6-4ae6-b394-1a8b603ea1d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:12.307685 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8c73a-42d2-4260-9cf2-5c5d29662e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:12.323412 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24959df-e3d6-4ae6-b394-1a8b603ea1d4 service nova] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:12.331144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5be8c73a-42d2-4260-9cf2-5c5d29662e8d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:12.332666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5be8c73a-42d2-4260-9cf2-5c5d29662e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:12.399304 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a548d-848e-45f9-8313-987a46e4b9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:12.400396 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a548d-848e-45f9-8313-987a46e4b9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:12.400693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a548d-848e-45f9-8313-987a46e4b9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:12.400974 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a548d-848e-45f9-8313-987a46e4b9eb admin admin] PATH_INFO: `/v3/projects/5df94e79b6fb46da9ed7ddeb74bde5f4` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:12.402698 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a548d-848e-45f9-8313-987a46e4b9eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df94e79b6fb46da9ed7ddeb74bde5f4)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:12.404726 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a548d-848e-45f9-8313-987a46e4b9eb admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:12.413925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 912/1832] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:12.432138 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdcbd0-ff4d-421c-a6d9-185a37c175db None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:12.432986 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdcbd0-ff4d-421c-a6d9-185a37c175db None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:12.433276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdcbd0-ff4d-421c-a6d9-185a37c175db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:12.464506 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-43bdcbd0-ff4d-421c-a6d9-185a37c175db None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:12.466117 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-43bdcbd0-ff4d-421c-a6d9-185a37c175db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:12.475612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 913/1833] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:12.479466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdcbd0-ff4d-421c-a6d9-185a37c175db None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:12.486875 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1f0a548d-848e-45f9-8313-987a46e4b9eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df94e79b6fb46da9ed7ddeb74bde5f4', 'request_id': 'req-1f0a548d-848e-45f9-8313-987a46e4b9eb'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:12.489820 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdcbd0-ff4d-421c-a6d9-185a37c175db None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:12.511311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1f0a548d-848e-45f9-8313-987a46e4b9eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df94e79b6fb46da9ed7ddeb74bde5f4', 'request_id': 'req-1f0a548d-848e-45f9-8313-987a46e4b9eb'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:12.522551 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1f0a548d-848e-45f9-8313-987a46e4b9eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df94e79b6fb46da9ed7ddeb74bde5f4', 'request_id': 'req-1f0a548d-848e-45f9-8313-987a46e4b9eb'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:12.556140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1f0a548d-848e-45f9-8313-987a46e4b9eb admin admin] The token cache is being invalidate because project 5df94e79b6fb46da9ed7ddeb74bde5f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:12.556614 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1f0a548d-848e-45f9-8313-987a46e4b9eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:12.557341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1f0a548d-848e-45f9-8313-987a46e4b9eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:12.560448 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 921/1834] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:12 2022] DELETE /identity/v3/projects/5df94e79b6fb46da9ed7ddeb74bde5f4 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:12.565004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c88a2f-7d43-429e-8a32-1bdb28798a1b None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:12.565371 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c88a2f-7d43-429e-8a32-1bdb28798a1b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:12.565725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c88a2f-7d43-429e-8a32-1bdb28798a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:12.590367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a8c88a2f-7d43-429e-8a32-1bdb28798a1b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:12.591342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a8c88a2f-7d43-429e-8a32-1bdb28798a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:12.612012 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 922/1835] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:12.620460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55d774-a865-40a5-8156-0dc27735a234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:12.621253 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55d774-a865-40a5-8156-0dc27735a234 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:12.621418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff840a9f-00f0-4af9-9465-28ae4fec67c5 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:12.621800 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55d774-a865-40a5-8156-0dc27735a234 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:12.622666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55d774-a865-40a5-8156-0dc27735a234 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:12.623073 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff840a9f-00f0-4af9-9465-28ae4fec67c5 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:12.623576 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff840a9f-00f0-4af9-9465-28ae4fec67c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:12.624206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55d774-a865-40a5-8156-0dc27735a234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:12.625856 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55d774-a865-40a5-8156-0dc27735a234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:12.629220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55d774-a865-40a5-8156-0dc27735a234 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:12.650640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-ff840a9f-00f0-4af9-9465-28ae4fec67c5 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:12.652508 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ff840a9f-00f0-4af9-9465-28ae4fec67c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:12.654944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 914/1836] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:52:12.659068 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55d774-a865-40a5-8156-0dc27735a234 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:12.663261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-df55d774-a865-40a5-8156-0dc27735a234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:12.670337 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 923/1837] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:12.683777 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff840a9f-00f0-4af9-9465-28ae4fec67c5 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:12.693796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff840a9f-00f0-4af9-9465-28ae4fec67c5 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:12.713359 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70335f-daf7-4c61-bbec-bba20d6a596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:12.714924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70335f-daf7-4c61-bbec-bba20d6a596e admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:12.716797 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70335f-daf7-4c61-bbec-bba20d6a596e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:12.717543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70335f-daf7-4c61-bbec-bba20d6a596e admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:12.719614 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70335f-daf7-4c61-bbec-bba20d6a596e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:12.720742 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70335f-daf7-4c61-bbec-bba20d6a596e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:12.720873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55d774-a865-40a5-8156-0dc27735a234 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:12.724212 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70335f-daf7-4c61-bbec-bba20d6a596e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:12.732799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce96ab2-f87a-4003-859e-61da793fcac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:12.733626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce96ab2-f87a-4003-859e-61da793fcac8 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:12.734058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce96ab2-f87a-4003-859e-61da793fcac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:12.734510 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce96ab2-f87a-4003-859e-61da793fcac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:12.735628 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce96ab2-f87a-4003-859e-61da793fcac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:12.739175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce96ab2-f87a-4003-859e-61da793fcac8 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:12.756242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 924/1838] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:12.758023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70335f-daf7-4c61-bbec-bba20d6a596e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:12.776320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70335f-daf7-4c61-bbec-bba20d6a596e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:12.787673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ae876-5b08-4ca6-b2d7-184d6fb72505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:12.788416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ae876-5b08-4ca6-b2d7-184d6fb72505 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:12.788760 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ae876-5b08-4ca6-b2d7-184d6fb72505 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:12.788978 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ae876-5b08-4ca6-b2d7-184d6fb72505 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:12.789730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ae876-5b08-4ca6-b2d7-184d6fb72505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:12.792170 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ae876-5b08-4ca6-b2d7-184d6fb72505 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:12.794557 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 915/1839] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:12 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:12.799398 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce96ab2-f87a-4003-859e-61da793fcac8 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:12.862762 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 925/1840] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:12 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:12.863555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ae876-5b08-4ca6-b2d7-184d6fb72505 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:12.922565 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce96ab2-f87a-4003-859e-61da793fcac8 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:13.024836 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ae876-5b08-4ca6-b2d7-184d6fb72505 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:13.030595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543b2c7-5475-4cec-8a56-b5b0532f403d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:13.031608 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543b2c7-5475-4cec-8a56-b5b0532f403d admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.032134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543b2c7-5475-4cec-8a56-b5b0532f403d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.032676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543b2c7-5475-4cec-8a56-b5b0532f403d admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.033523 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543b2c7-5475-4cec-8a56-b5b0532f403d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:13.038122 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543b2c7-5475-4cec-8a56-b5b0532f403d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:13.060549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345a310-ed62-4741-91e1-a7438c438bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:13.061224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a310-ed62-4741-91e1-a7438c438bea admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.061416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a310-ed62-4741-91e1-a7438c438bea admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.064698 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a310-ed62-4741-91e1-a7438c438bea admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.065489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345a310-ed62-4741-91e1-a7438c438bea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:13.072343 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345a310-ed62-4741-91e1-a7438c438bea admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:13.102372 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 926/1841] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:12 2022] POST /identity/v3/users => generated 284 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:13.107570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345a310-ed62-4741-91e1-a7438c438bea admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:13.115700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c345a310-ed62-4741-91e1-a7438c438bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:13.115881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 916/1842] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:12 2022] POST /identity/v3/users => generated 294 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:13.119970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543b2c7-5475-4cec-8a56-b5b0532f403d admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:13.146742 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345a310-ed62-4741-91e1-a7438c438bea admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:13.150173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543b2c7-5475-4cec-8a56-b5b0532f403d admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:13.154580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:13.155626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.155847 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.156079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] PATH_INFO: `/v3/projects/04e0137f927943859f6a355b37be9dd0` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.157604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] RBAC: Authorizing `identity:delete_project(project_id=04e0137f927943859f6a355b37be9dd0)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:13.164396 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:13.167521 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bf269-37a1-4adc-9821-1edf9011f994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:13.168502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bf269-37a1-4adc-9821-1edf9011f994 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.169551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bf269-37a1-4adc-9821-1edf9011f994 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.170263 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bf269-37a1-4adc-9821-1edf9011f994 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.171687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724bf269-37a1-4adc-9821-1edf9011f994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:13.174068 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724bf269-37a1-4adc-9821-1edf9011f994 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:13.207211 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 917/1843] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:13.208695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bf269-37a1-4adc-9821-1edf9011f994 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:13.232132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bf269-37a1-4adc-9821-1edf9011f994 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:13.254381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04e0137f927943859f6a355b37be9dd0', 'request_id': 'req-ad9924a5-4809-4f2b-8955-02bfd4c2445f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:13.265761 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04e0137f927943859f6a355b37be9dd0', 'request_id': 'req-ad9924a5-4809-4f2b-8955-02bfd4c2445f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:13.277975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04e0137f927943859f6a355b37be9dd0', 'request_id': 'req-ad9924a5-4809-4f2b-8955-02bfd4c2445f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:13.298072 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] The token cache is being invalidate because project 04e0137f927943859f6a355b37be9dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:13.298544 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:13.299532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:13.301856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 927/1844] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:13 2022] DELETE /identity/v3/projects/04e0137f927943859f6a355b37be9dd0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:13.302763 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:13.304245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:13.335666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234751ae-8bb3-43a5-848d-5e3c938a151b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:13.336342 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234751ae-8bb3-43a5-848d-5e3c938a151b admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.336664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234751ae-8bb3-43a5-848d-5e3c938a151b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.336975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234751ae-8bb3-43a5-848d-5e3c938a151b admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.337702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234751ae-8bb3-43a5-848d-5e3c938a151b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:13.339558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234751ae-8bb3-43a5-848d-5e3c938a151b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:13.352350 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 918/1845] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:13.353230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234751ae-8bb3-43a5-848d-5e3c938a151b admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:13.355273 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-234751ae-8bb3-43a5-848d-5e3c938a151b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:13.369462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234751ae-8bb3-43a5-848d-5e3c938a151b admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:13.375887 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e2bfd-d987-4eb0-80d0-931947947f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:13.376640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e2bfd-d987-4eb0-80d0-931947947f7b admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.376999 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e2bfd-d987-4eb0-80d0-931947947f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.377385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e2bfd-d987-4eb0-80d0-931947947f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.378188 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e2bfd-d987-4eb0-80d0-931947947f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:13.379913 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e2bfd-d987-4eb0-80d0-931947947f7b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:13.386574 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9924a5-4809-4f2b-8955-02bfd4c2445f admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:13.392210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:13.393235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.393358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.393933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] PATH_INFO: `/v3/projects/8d52b78bce534d219ac697c03c95d230` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.394334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 919/1846] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:13.396805 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af081e9c-1d57-48e8-acf4-74d26ee165df None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.397325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af081e9c-1d57-48e8-acf4-74d26ee165df None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.397794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af081e9c-1d57-48e8-acf4-74d26ee165df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.399137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d52b78bce534d219ac697c03c95d230)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:13.401418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:13.422448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-af081e9c-1d57-48e8-acf4-74d26ee165df None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:13.424325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-af081e9c-1d57-48e8-acf4-74d26ee165df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:13.439815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 920/1847] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:13.441467 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af081e9c-1d57-48e8-acf4-74d26ee165df None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:13.447227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-af081e9c-1d57-48e8-acf4-74d26ee165df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:13.462141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af081e9c-1d57-48e8-acf4-74d26ee165df None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:13.470173 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d52b78bce534d219ac697c03c95d230', 'request_id': 'req-dbef3af4-13fe-490f-914b-45fb618b9371'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:13.476939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d52b78bce534d219ac697c03c95d230', 'request_id': 'req-dbef3af4-13fe-490f-914b-45fb618b9371'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:13.493260 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d52b78bce534d219ac697c03c95d230', 'request_id': 'req-dbef3af4-13fe-490f-914b-45fb618b9371'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:13.506560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] The token cache is being invalidate because project 8d52b78bce534d219ac697c03c95d230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:13.507001 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:13.511579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:13.514110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 928/1848] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:13 2022] DELETE /identity/v3/projects/8d52b78bce534d219ac697c03c95d230 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:13.515361 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:13.518362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:13.598929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bbdbd-3e49-4860-8f01-7942f466d4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:13.599857 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bbdbd-3e49-4860-8f01-7942f466d4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.600299 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bbdbd-3e49-4860-8f01-7942f466d4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.600759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bbdbd-3e49-4860-8f01-7942f466d4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.601830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef3af4-13fe-490f-914b-45fb618b9371 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:13.602813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bbdbd-3e49-4860-8f01-7942f466d4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:13.607660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebf634-38c3-437e-8c86-dea74206d9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:13.608624 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebf634-38c3-437e-8c86-dea74206d9c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.608994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebf634-38c3-437e-8c86-dea74206d9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.609335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebf634-38c3-437e-8c86-dea74206d9c5 admin admin] PATH_INFO: `/v3/projects/41aa127145914e8a810abd619028e8c0/users/ca34b0a71d554031bf6b8439e0248795/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.612279 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bbdbd-3e49-4860-8f01-7942f466d4e8 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:13.629721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebf634-38c3-437e-8c86-dea74206d9c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41aa127145914e8a810abd619028e8c0, user_id=ca34b0a71d554031bf6b8439e0248795, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:13.633273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebf634-38c3-437e-8c86-dea74206d9c5 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:13.640789 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 921/1849] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:13.643134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b1656-43da-4103-a653-226c722a0cbd None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.643693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b1656-43da-4103-a653-226c722a0cbd None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.644150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b1656-43da-4103-a653-226c722a0cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.657966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 929/1850] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:13 2022] PUT /identity/v3/projects/41aa127145914e8a810abd619028e8c0/users/ca34b0a71d554031bf6b8439e0248795/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:13.659317 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892a45a-ad78-40c4-982a-36a443e05340 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.659608 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892a45a-ad78-40c4-982a-36a443e05340 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.660877 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892a45a-ad78-40c4-982a-36a443e05340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.675866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-784b1656-43da-4103-a653-226c722a0cbd None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:13.677078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-784b1656-43da-4103-a653-226c722a0cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:13.687602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-5892a45a-ad78-40c4-982a-36a443e05340 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:13.688959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-5892a45a-ad78-40c4-982a-36a443e05340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:13.753514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 922/1851] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:13.754422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b1656-43da-4103-a653-226c722a0cbd None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:13.755730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-784b1656-43da-4103-a653-226c722a0cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:13.767775 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b1656-43da-4103-a653-226c722a0cbd None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:13.773047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455cf8e-d063-4c0e-a72c-fd237c12232e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:13.774105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455cf8e-d063-4c0e-a72c-fd237c12232e admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.774669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455cf8e-d063-4c0e-a72c-fd237c12232e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.774921 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 930/1852] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:13.775293 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455cf8e-d063-4c0e-a72c-fd237c12232e admin admin] PATH_INFO: `/v3/projects/fd2f8917ab214de99e83e179a1c9abcd/users/9ad267aebd3f44c981291a0d8305b865/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.776667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676eab35-5413-4aaf-92f6-37feb6353bbb None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.777051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676eab35-5413-4aaf-92f6-37feb6353bbb None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.777713 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676eab35-5413-4aaf-92f6-37feb6353bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.790922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455cf8e-d063-4c0e-a72c-fd237c12232e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd2f8917ab214de99e83e179a1c9abcd, user_id=9ad267aebd3f44c981291a0d8305b865, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:13.793524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455cf8e-d063-4c0e-a72c-fd237c12232e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:13.807235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-676eab35-5413-4aaf-92f6-37feb6353bbb None None] MFA Rules not processed for user `ca34b0a71d554031bf6b8439e0248795`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:13.807724 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 923/1853] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:13 2022] PUT /identity/v3/projects/fd2f8917ab214de99e83e179a1c9abcd/users/9ad267aebd3f44c981291a0d8305b865/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:13.808518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455cf8e-d063-4c0e-a72c-fd237c12232e admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:13.808649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-676eab35-5413-4aaf-92f6-37feb6353bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:13.872387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 931/1854] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:13.874690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455cf8e-d063-4c0e-a72c-fd237c12232e admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:13.876150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5504ae-502e-4335-a504-c7ddeae7de57 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.876573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5504ae-502e-4335-a504-c7ddeae7de57 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.876954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5504ae-502e-4335-a504-c7ddeae7de57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.879566 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e862bc4-0bdf-4202-942f-830d2a31c667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:13.880219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e862bc4-0bdf-4202-942f-830d2a31c667 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.880504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e862bc4-0bdf-4202-942f-830d2a31c667 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.880782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e862bc4-0bdf-4202-942f-830d2a31c667 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.881501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e862bc4-0bdf-4202-942f-830d2a31c667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:13.881807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e862bc4-0bdf-4202-942f-830d2a31c667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:13.883472 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e862bc4-0bdf-4202-942f-830d2a31c667 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:13.898842 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 924/1855] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:13.898842 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811a56e-92ca-44fd-ba73-ab8eb427ab60 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.898842 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811a56e-92ca-44fd-ba73-ab8eb427ab60 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.898842 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811a56e-92ca-44fd-ba73-ab8eb427ab60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.899198 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-2b5504ae-502e-4335-a504-c7ddeae7de57 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:13.900468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-2b5504ae-502e-4335-a504-c7ddeae7de57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:13.909095 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 932/1856] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:13.909871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5504ae-502e-4335-a504-c7ddeae7de57 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:13.919628 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5504ae-502e-4335-a504-c7ddeae7de57 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:13.924234 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918982a-9259-4da4-982a-8b76b6ca2b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:13.924873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918982a-9259-4da4-982a-8b76b6ca2b00 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:13.925155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918982a-9259-4da4-982a-8b76b6ca2b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:13.925429 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918982a-9259-4da4-982a-8b76b6ca2b00 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:13.926106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918982a-9259-4da4-982a-8b76b6ca2b00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:13.926467 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-1811a56e-92ca-44fd-ba73-ab8eb427ab60 None None] MFA Rules not processed for user `9ad267aebd3f44c981291a0d8305b865`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:13.927703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-1811a56e-92ca-44fd-ba73-ab8eb427ab60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:13.928034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918982a-9259-4da4-982a-8b76b6ca2b00 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:13.952988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 933/1857] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:13 2022] POST /identity/v3/projects => generated 369 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:13.954003 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918982a-9259-4da4-982a-8b76b6ca2b00 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.005207 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 925/1858] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:14.006417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811a56e-92ca-44fd-ba73-ab8eb427ab60 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.017556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811a56e-92ca-44fd-ba73-ab8eb427ab60 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.029518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918982a-9259-4da4-982a-8b76b6ca2b00 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.041564 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0e1a3-bba1-488b-af79-cb27059f0096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.043062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0e1a3-bba1-488b-af79-cb27059f0096 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.043732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0e1a3-bba1-488b-af79-cb27059f0096 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.044331 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0e1a3-bba1-488b-af79-cb27059f0096 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.045773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0e1a3-bba1-488b-af79-cb27059f0096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.048635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0e1a3-bba1-488b-af79-cb27059f0096 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.094000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec49af-4ca7-4b75-9f1e-6c3ebfb93e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.094813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec49af-4ca7-4b75-9f1e-6c3ebfb93e4e admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.095102 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec49af-4ca7-4b75-9f1e-6c3ebfb93e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.095378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec49af-4ca7-4b75-9f1e-6c3ebfb93e4e admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.096159 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec49af-4ca7-4b75-9f1e-6c3ebfb93e4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:14.096470 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec49af-4ca7-4b75-9f1e-6c3ebfb93e4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.098184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec49af-4ca7-4b75-9f1e-6c3ebfb93e4e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.103523 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 934/1859] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:13 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:14.104559 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0e1a3-bba1-488b-af79-cb27059f0096 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.113249 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 926/1860] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:14.118057 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec49af-4ca7-4b75-9f1e-6c3ebfb93e4e admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.129015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec49af-4ca7-4b75-9f1e-6c3ebfb93e4e admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.186692 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0e1a3-bba1-488b-af79-cb27059f0096 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.198282 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af432bd4-8189-40bd-86c3-085348468166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.199442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af432bd4-8189-40bd-86c3-085348468166 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.200667 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af432bd4-8189-40bd-86c3-085348468166 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.201113 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af432bd4-8189-40bd-86c3-085348468166 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.201795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b47b35-ae7e-49b8-8bae-0751ca33d544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.202337 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af432bd4-8189-40bd-86c3-085348468166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.203419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47b35-ae7e-49b8-8bae-0751ca33d544 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.203774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47b35-ae7e-49b8-8bae-0751ca33d544 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.204115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47b35-ae7e-49b8-8bae-0751ca33d544 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.204887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b47b35-ae7e-49b8-8bae-0751ca33d544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.205204 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af432bd4-8189-40bd-86c3-085348468166 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.207101 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b47b35-ae7e-49b8-8bae-0751ca33d544 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.236899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 935/1861] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:14 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:14.236899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b47b35-ae7e-49b8-8bae-0751ca33d544 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.243893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 927/1862] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:14 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:14.245168 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af432bd4-8189-40bd-86c3-085348468166 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.274656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b47b35-ae7e-49b8-8bae-0751ca33d544 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.318502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af432bd4-8189-40bd-86c3-085348468166 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.332346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1b6a9-c689-4eb5-a961-f642faf92217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.333188 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1b6a9-c689-4eb5-a961-f642faf92217 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.333614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1b6a9-c689-4eb5-a961-f642faf92217 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.333903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1b6a9-c689-4eb5-a961-f642faf92217 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.335573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1b6a9-c689-4eb5-a961-f642faf92217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.338164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1b6a9-c689-4eb5-a961-f642faf92217 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.379702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 928/1863] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:14 2022] POST /identity/v3/projects => generated 393 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:14.382980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1b6a9-c689-4eb5-a961-f642faf92217 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.391198 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d14bb-0bb2-48bf-8bf9-9377f8516c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.391979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d14bb-0bb2-48bf-8bf9-9377f8516c0b admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.392355 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d14bb-0bb2-48bf-8bf9-9377f8516c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.392874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d14bb-0bb2-48bf-8bf9-9377f8516c0b admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.394289 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d14bb-0bb2-48bf-8bf9-9377f8516c0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.395975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d14bb-0bb2-48bf-8bf9-9377f8516c0b admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.419797 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 936/1864] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:14 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:14.420808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d14bb-0bb2-48bf-8bf9-9377f8516c0b admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.433349 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d14bb-0bb2-48bf-8bf9-9377f8516c0b admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.484410 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1b6a9-c689-4eb5-a961-f642faf92217 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.492366 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2569bc-8093-4066-b148-02b7cc213515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.493111 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2569bc-8093-4066-b148-02b7cc213515 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.493408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2569bc-8093-4066-b148-02b7cc213515 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.493689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2569bc-8093-4066-b148-02b7cc213515 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.494432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2569bc-8093-4066-b148-02b7cc213515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.498004 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2569bc-8093-4066-b148-02b7cc213515 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.499199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3352ec-e8d7-4164-b2b7-cfe0f4ec187c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.500705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3352ec-e8d7-4164-b2b7-cfe0f4ec187c admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.500705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3352ec-e8d7-4164-b2b7-cfe0f4ec187c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.500705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3352ec-e8d7-4164-b2b7-cfe0f4ec187c admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.501425 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3352ec-e8d7-4164-b2b7-cfe0f4ec187c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.503745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3352ec-e8d7-4164-b2b7-cfe0f4ec187c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.525745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 929/1865] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:14.526572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2569bc-8093-4066-b148-02b7cc213515 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.540688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2569bc-8093-4066-b148-02b7cc213515 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.540832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 937/1866] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:14 2022] POST /identity/v3/users => generated 300 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:14.543007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3352ec-e8d7-4164-b2b7-cfe0f4ec187c admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.547452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c01847-3dbc-407e-a20d-b863c08f207d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.548710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c01847-3dbc-407e-a20d-b863c08f207d admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.549181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c01847-3dbc-407e-a20d-b863c08f207d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.549690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c01847-3dbc-407e-a20d-b863c08f207d admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.550686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c01847-3dbc-407e-a20d-b863c08f207d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.553799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c01847-3dbc-407e-a20d-b863c08f207d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.569493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3352ec-e8d7-4164-b2b7-cfe0f4ec187c admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.583110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf48e7b-36ed-4e41-bf5c-282142bbc3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.584007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf48e7b-36ed-4e41-bf5c-282142bbc3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.584326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf48e7b-36ed-4e41-bf5c-282142bbc3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.586218 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf48e7b-36ed-4e41-bf5c-282142bbc3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.586557 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 930/1867] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:14 2022] POST /identity/v3/users => generated 297 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:14.587721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c01847-3dbc-407e-a20d-b863c08f207d admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.591784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf48e7b-36ed-4e41-bf5c-282142bbc3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.594272 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf48e7b-36ed-4e41-bf5c-282142bbc3b7 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.600420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c01847-3dbc-407e-a20d-b863c08f207d admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.612741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc99b04-bb90-4e80-b4b1-993620692b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.614029 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc99b04-bb90-4e80-b4b1-993620692b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.614759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc99b04-bb90-4e80-b4b1-993620692b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.615566 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc99b04-bb90-4e80-b4b1-993620692b68 admin admin] PATH_INFO: `/v3/projects/726370958a6e43c7aaca0fc3bf72759d/users/da643d51395a495494008c5b8a81d9d1/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.623395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 938/1868] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:14.624661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf48e7b-36ed-4e41-bf5c-282142bbc3b7 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.645357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf48e7b-36ed-4e41-bf5c-282142bbc3b7 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.647031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc99b04-bb90-4e80-b4b1-993620692b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=726370958a6e43c7aaca0fc3bf72759d, user_id=da643d51395a495494008c5b8a81d9d1, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.650607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc99b04-bb90-4e80-b4b1-993620692b68 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.666558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04b7f2-b215-420c-8a16-266c1321493d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.666905 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 931/1869] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:14 2022] PUT /identity/v3/projects/726370958a6e43c7aaca0fc3bf72759d/users/da643d51395a495494008c5b8a81d9d1/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:14.666994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04b7f2-b215-420c-8a16-266c1321493d admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.667536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04b7f2-b215-420c-8a16-266c1321493d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.667536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04b7f2-b215-420c-8a16-266c1321493d admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.672399 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04b7f2-b215-420c-8a16-266c1321493d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.672885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc99b04-bb90-4e80-b4b1-993620692b68 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.674064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04b7f2-b215-420c-8a16-266c1321493d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.709746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 939/1870] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:14.710745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04b7f2-b215-420c-8a16-266c1321493d admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.732221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04b7f2-b215-420c-8a16-266c1321493d admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.806880 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc99b04-bb90-4e80-b4b1-993620692b68 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.813538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564e5f5-bddc-4143-96a0-1c5297627e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.814446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564e5f5-bddc-4143-96a0-1c5297627e56 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.815556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564e5f5-bddc-4143-96a0-1c5297627e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.816205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564e5f5-bddc-4143-96a0-1c5297627e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.817806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564e5f5-bddc-4143-96a0-1c5297627e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.819980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564e5f5-bddc-4143-96a0-1c5297627e56 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.854290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8105d-dbbb-4fee-9e8b-22c7c56d3d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.855191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8105d-dbbb-4fee-9e8b-22c7c56d3d88 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.855482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8105d-dbbb-4fee-9e8b-22c7c56d3d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.857154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8105d-dbbb-4fee-9e8b-22c7c56d3d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.857940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8105d-dbbb-4fee-9e8b-22c7c56d3d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.858237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 932/1871] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:14.860739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564e5f5-bddc-4143-96a0-1c5297627e56 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.861437 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8105d-dbbb-4fee-9e8b-22c7c56d3d88 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.886442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 940/1872] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:14.887404 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8105d-dbbb-4fee-9e8b-22c7c56d3d88 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.895752 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564e5f5-bddc-4143-96a0-1c5297627e56 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.912509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8105d-dbbb-4fee-9e8b-22c7c56d3d88 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.930446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860a76b-8232-4c25-9d40-447093964f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.930601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fbc4e-e3dd-4312-9cdc-735787385d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.931160 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860a76b-8232-4c25-9d40-447093964f11 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.931431 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fbc4e-e3dd-4312-9cdc-735787385d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.931544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860a76b-8232-4c25-9d40-447093964f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.931708 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860a76b-8232-4c25-9d40-447093964f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.931810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fbc4e-e3dd-4312-9cdc-735787385d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.931990 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fbc4e-e3dd-4312-9cdc-735787385d4f admin admin] PATH_INFO: `/v3/projects/f6e750f9f47d4acc9f02050d068a3d7a/users/2376cc2301824dcfa8e7550c7949f434/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.932424 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860a76b-8232-4c25-9d40-447093964f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.934125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860a76b-8232-4c25-9d40-447093964f11 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.955407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 933/1873] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:14.956497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860a76b-8232-4c25-9d40-447093964f11 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:14.970136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860a76b-8232-4c25-9d40-447093964f11 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:14.974141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fbc4e-e3dd-4312-9cdc-735787385d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e750f9f47d4acc9f02050d068a3d7a, user_id=2376cc2301824dcfa8e7550c7949f434, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.975403 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fbc4e-e3dd-4312-9cdc-735787385d4f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:14.978443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9f503-e552-45f8-a28d-641f8ca793df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:14.979395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9f503-e552-45f8-a28d-641f8ca793df admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:14.979869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9f503-e552-45f8-a28d-641f8ca793df admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:14.980278 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9f503-e552-45f8-a28d-641f8ca793df admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:14.981247 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9f503-e552-45f8-a28d-641f8ca793df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:14.983725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9f503-e552-45f8-a28d-641f8ca793df admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:15.003404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 934/1874] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:15.004325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 941/1875] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:14 2022] PUT /identity/v3/projects/f6e750f9f47d4acc9f02050d068a3d7a/users/2376cc2301824dcfa8e7550c7949f434/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:15.005383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fbc4e-e3dd-4312-9cdc-735787385d4f admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:15.006980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9f503-e552-45f8-a28d-641f8ca793df admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:15.017635 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9f503-e552-45f8-a28d-641f8ca793df admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:15.098894 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fbc4e-e3dd-4312-9cdc-735787385d4f admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:15.104594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbed22-6c51-4df7-a289-cb5569d9b2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:15.105348 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbed22-6c51-4df7-a289-cb5569d9b2f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:15.105641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbed22-6c51-4df7-a289-cb5569d9b2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:15.105922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbed22-6c51-4df7-a289-cb5569d9b2f8 admin admin] PATH_INFO: `/v3/projects/726370958a6e43c7aaca0fc3bf72759d/users/da643d51395a495494008c5b8a81d9d1/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:15.108800 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbed22-6c51-4df7-a289-cb5569d9b2f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=726370958a6e43c7aaca0fc3bf72759d, user_id=da643d51395a495494008c5b8a81d9d1, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:15.111775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbed22-6c51-4df7-a289-cb5569d9b2f8 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:15.133537 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 942/1876] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:15 2022] PUT /identity/v3/projects/726370958a6e43c7aaca0fc3bf72759d/users/da643d51395a495494008c5b8a81d9d1/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:15.134778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbed22-6c51-4df7-a289-cb5569d9b2f8 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:15.155296 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81d94f-a8ce-4f46-a70a-0480808bab23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:15.156303 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81d94f-a8ce-4f46-a70a-0480808bab23 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:15.156501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81d94f-a8ce-4f46-a70a-0480808bab23 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:15.157240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81d94f-a8ce-4f46-a70a-0480808bab23 admin admin] PATH_INFO: `/v3/projects/76db9f5b2dea49f8b11294cd98fc11ce/users/8c730524e557459db124c88b0e46e267/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:15.187635 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81d94f-a8ce-4f46-a70a-0480808bab23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76db9f5b2dea49f8b11294cd98fc11ce, user_id=8c730524e557459db124c88b0e46e267, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:15.190873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81d94f-a8ce-4f46-a70a-0480808bab23 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:15.207390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 935/1877] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:15 2022] PUT /identity/v3/projects/76db9f5b2dea49f8b11294cd98fc11ce/users/8c730524e557459db124c88b0e46e267/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:15.208203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81d94f-a8ce-4f46-a70a-0480808bab23 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:15.220295 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbed22-6c51-4df7-a289-cb5569d9b2f8 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:15.336473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81d94f-a8ce-4f46-a70a-0480808bab23 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:15.344408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4d206-1272-4938-8a77-2c0287cf0d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:15.344942 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d206-1272-4938-8a77-2c0287cf0d9e admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:15.345278 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d206-1272-4938-8a77-2c0287cf0d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:15.345558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d206-1272-4938-8a77-2c0287cf0d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:15.346483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4d206-1272-4938-8a77-2c0287cf0d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:15.348586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4d206-1272-4938-8a77-2c0287cf0d9e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:15.361851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 936/1878] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:15.362034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938eac11-1c51-412c-9b5e-5c83bb2b3614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:15.363235 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e8fe5-a4b4-4883-9c3e-93e40ca2b6ef None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:15.363531 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e8fe5-a4b4-4883-9c3e-93e40ca2b6ef None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:15.363813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e8fe5-a4b4-4883-9c3e-93e40ca2b6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:15.364091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938eac11-1c51-412c-9b5e-5c83bb2b3614 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:15.364384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938eac11-1c51-412c-9b5e-5c83bb2b3614 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:15.365578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938eac11-1c51-412c-9b5e-5c83bb2b3614 admin admin] PATH_INFO: `/v3/projects/74f4bdc06a674d66bbe5f647e68afaa2/users/ac8dbcfb37ee4548b1403442284df411/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:15.396033 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938eac11-1c51-412c-9b5e-5c83bb2b3614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f4bdc06a674d66bbe5f647e68afaa2, user_id=ac8dbcfb37ee4548b1403442284df411, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:15.397255 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-a97e8fe5-a4b4-4883-9c3e-93e40ca2b6ef None None] MFA Rules not processed for user `da643d51395a495494008c5b8a81d9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:15.398583 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a97e8fe5-a4b4-4883-9c3e-93e40ca2b6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:15.399185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938eac11-1c51-412c-9b5e-5c83bb2b3614 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:15.433872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 943/1879] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:15 2022] PUT /identity/v3/projects/74f4bdc06a674d66bbe5f647e68afaa2/users/ac8dbcfb37ee4548b1403442284df411/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:15.433872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85740c1-a31b-48fb-af3e-5a44c7bd16e3 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:15.433872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85740c1-a31b-48fb-af3e-5a44c7bd16e3 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:15.433872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85740c1-a31b-48fb-af3e-5a44c7bd16e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:15.487442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-f85740c1-a31b-48fb-af3e-5a44c7bd16e3 None None] MFA Rules not processed for user `8c730524e557459db124c88b0e46e267`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:15.489799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f85740c1-a31b-48fb-af3e-5a44c7bd16e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:15.536164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 937/1880] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:15 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:15.537018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e8fe5-a4b4-4883-9c3e-93e40ca2b6ef None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:15.567700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e8fe5-a4b4-4883-9c3e-93e40ca2b6ef None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:15.607737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 944/1881] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:15.609478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c17e3-1eb4-446b-b37c-87575ca39d4a None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:15.609771 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c17e3-1eb4-446b-b37c-87575ca39d4a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:15.610099 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c17e3-1eb4-446b-b37c-87575ca39d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:15.650278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-6f5c17e3-1eb4-446b-b37c-87575ca39d4a None None] MFA Rules not processed for user `ac8dbcfb37ee4548b1403442284df411`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:15.651496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-6f5c17e3-1eb4-446b-b37c-87575ca39d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:15.680526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffe94c-bea1-485b-8bb1-8920acf73697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:15.681510 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffe94c-bea1-485b-8bb1-8920acf73697 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:15.681794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffe94c-bea1-485b-8bb1-8920acf73697 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:15.682101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffe94c-bea1-485b-8bb1-8920acf73697 admin admin] PATH_INFO: `/v3/projects/f6e750f9f47d4acc9f02050d068a3d7a/users/2376cc2301824dcfa8e7550c7949f434/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:15.684135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffe94c-bea1-485b-8bb1-8920acf73697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e750f9f47d4acc9f02050d068a3d7a, user_id=2376cc2301824dcfa8e7550c7949f434, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:15.686977 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffe94c-bea1-485b-8bb1-8920acf73697 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:15.699027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 938/1882] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:15 2022] PUT /identity/v3/projects/f6e750f9f47d4acc9f02050d068a3d7a/users/2376cc2301824dcfa8e7550c7949f434/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:15.700885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffe94c-bea1-485b-8bb1-8920acf73697 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:15.702920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-afffe94c-bea1-485b-8bb1-8920acf73697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:15.758791 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 945/1883] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:15.760283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c17e3-1eb4-446b-b37c-87575ca39d4a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:15.777822 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffe94c-bea1-485b-8bb1-8920acf73697 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:15.782032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c17e3-1eb4-446b-b37c-87575ca39d4a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:15.788962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbef7e9-530e-47fd-a0dc-e868a15ec459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:15.791952 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbef7e9-530e-47fd-a0dc-e868a15ec459 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:15.791952 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbef7e9-530e-47fd-a0dc-e868a15ec459 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:15.791952 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbef7e9-530e-47fd-a0dc-e868a15ec459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:15.850160 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b166658-f871-4ad2-affd-20264ce198e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:15.850963 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b166658-f871-4ad2-affd-20264ce198e1 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:15.851319 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b166658-f871-4ad2-affd-20264ce198e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:15.851668 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b166658-f871-4ad2-affd-20264ce198e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:15.852436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b166658-f871-4ad2-affd-20264ce198e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:15.854428 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b166658-f871-4ad2-affd-20264ce198e1 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:15.866287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbef7e9-530e-47fd-a0dc-e868a15ec459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:15.867856 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbef7e9-530e-47fd-a0dc-e868a15ec459 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:15.878372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 939/1884] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:15 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:15.879694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a99704-7c70-4051-ac56-74717da5e6c3 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:15.880024 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a99704-7c70-4051-ac56-74717da5e6c3 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:15.880478 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a99704-7c70-4051-ac56-74717da5e6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:15.884295 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 946/1885] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:15 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:15.884295 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b166658-f871-4ad2-affd-20264ce198e1 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:15.910435 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-77a99704-7c70-4051-ac56-74717da5e6c3 None None] MFA Rules not processed for user `2376cc2301824dcfa8e7550c7949f434`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:15.911708 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-77a99704-7c70-4051-ac56-74717da5e6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:15.951791 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b166658-f871-4ad2-affd-20264ce198e1 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:15.957438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a6cae-e06c-4ea1-8d8b-a4882d5b2b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:15.958143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a6cae-e06c-4ea1-8d8b-a4882d5b2b0e admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:15.958432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a6cae-e06c-4ea1-8d8b-a4882d5b2b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:15.958727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a6cae-e06c-4ea1-8d8b-a4882d5b2b0e admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:15.959409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a6cae-e06c-4ea1-8d8b-a4882d5b2b0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:15.961325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a6cae-e06c-4ea1-8d8b-a4882d5b2b0e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:15.993534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 940/1886] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:15.993534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a99704-7c70-4051-ac56-74717da5e6c3 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:15.999716 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 947/1887] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:15 2022] POST /identity/v3/projects => generated 391 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:16.000800 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a6cae-e06c-4ea1-8d8b-a4882d5b2b0e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:16.002646 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-287a6cae-e06c-4ea1-8d8b-a4882d5b2b0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:16.013967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a99704-7c70-4051-ac56-74717da5e6c3 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:16.059149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a6cae-e06c-4ea1-8d8b-a4882d5b2b0e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:16.064166 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c420b9-e479-4646-9cf1-3f3d42f0d397 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:16.064810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c420b9-e479-4646-9cf1-3f3d42f0d397 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.065095 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c420b9-e479-4646-9cf1-3f3d42f0d397 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.065375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c420b9-e479-4646-9cf1-3f3d42f0d397 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.072436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde1ebd-7b3a-474f-9475-c2591919574a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:16.073107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde1ebd-7b3a-474f-9475-c2591919574a admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.073402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde1ebd-7b3a-474f-9475-c2591919574a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.073684 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde1ebd-7b3a-474f-9475-c2591919574a admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.074650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde1ebd-7b3a-474f-9475-c2591919574a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:16.077552 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde1ebd-7b3a-474f-9475-c2591919574a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:16.097450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 941/1888] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:15 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:16.098152 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde1ebd-7b3a-474f-9475-c2591919574a admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:16.107835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde1ebd-7b3a-474f-9475-c2591919574a admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:16.112604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e473919-0f30-4595-8c91-958d1ef18b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:16.113250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e473919-0f30-4595-8c91-958d1ef18b62 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.113540 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e473919-0f30-4595-8c91-958d1ef18b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.113822 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e473919-0f30-4595-8c91-958d1ef18b62 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.114508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e473919-0f30-4595-8c91-958d1ef18b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:16.116741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e473919-0f30-4595-8c91-958d1ef18b62 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:16.120740 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c420b9-e479-4646-9cf1-3f3d42f0d397 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:16.122719 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c420b9-e479-4646-9cf1-3f3d42f0d397 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:16.132788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 948/1889] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:16.133721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c420b9-e479-4646-9cf1-3f3d42f0d397 service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:16.140359 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 942/1890] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:16 2022] POST /identity/v3/users => generated 302 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:16.140359 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e473919-0f30-4595-8c91-958d1ef18b62 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:16.144488 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c420b9-e479-4646-9cf1-3f3d42f0d397 service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:16.149311 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e473919-0f30-4595-8c91-958d1ef18b62 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:16.149821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92534ac4-4509-48f6-8fa3-396f8f2bcc5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:16.150647 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92534ac4-4509-48f6-8fa3-396f8f2bcc5c service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.150973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92534ac4-4509-48f6-8fa3-396f8f2bcc5c service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.151251 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92534ac4-4509-48f6-8fa3-396f8f2bcc5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.158436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2be777-da58-4888-8969-6050b4a96fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:16.159140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2be777-da58-4888-8969-6050b4a96fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.159429 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2be777-da58-4888-8969-6050b4a96fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.159705 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2be777-da58-4888-8969-6050b4a96fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.160437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2be777-da58-4888-8969-6050b4a96fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:16.162394 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2be777-da58-4888-8969-6050b4a96fd1 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:16.175854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 943/1891] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:16.177109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2be777-da58-4888-8969-6050b4a96fd1 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:16.187841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2be777-da58-4888-8969-6050b4a96fd1 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:16.196450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12188fad-af4c-457c-b261-9229ad88c243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:16.197111 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12188fad-af4c-457c-b261-9229ad88c243 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.197395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12188fad-af4c-457c-b261-9229ad88c243 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.197672 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12188fad-af4c-457c-b261-9229ad88c243 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.198560 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12188fad-af4c-457c-b261-9229ad88c243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:16.200964 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12188fad-af4c-457c-b261-9229ad88c243 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:16.212786 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 944/1892] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:16.213672 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12188fad-af4c-457c-b261-9229ad88c243 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:16.223988 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12188fad-af4c-457c-b261-9229ad88c243 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:16.229746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af69bd-c0d4-4e26-8810-104f5d18f522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:16.230909 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af69bd-c0d4-4e26-8810-104f5d18f522 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.231831 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af69bd-c0d4-4e26-8810-104f5d18f522 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.232226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af69bd-c0d4-4e26-8810-104f5d18f522 admin admin] PATH_INFO: `/v3/projects/5be0cf2755284ed0b6016436593e2c71/users/8023f2252ff24e20bbb7c4ea4a89b24a/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.235888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92534ac4-4509-48f6-8fa3-396f8f2bcc5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:16.237397 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92534ac4-4509-48f6-8fa3-396f8f2bcc5c service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:16.249740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af69bd-c0d4-4e26-8810-104f5d18f522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5be0cf2755284ed0b6016436593e2c71, user_id=8023f2252ff24e20bbb7c4ea4a89b24a, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:16.252749 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af69bd-c0d4-4e26-8810-104f5d18f522 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:16.254900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 949/1893] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:16.255778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92534ac4-4509-48f6-8fa3-396f8f2bcc5c service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:16.268810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92534ac4-4509-48f6-8fa3-396f8f2bcc5c service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:16.270045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 945/1894] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:16 2022] PUT /identity/v3/projects/5be0cf2755284ed0b6016436593e2c71/users/8023f2252ff24e20bbb7c4ea4a89b24a/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:16.274038 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af69bd-c0d4-4e26-8810-104f5d18f522 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:16.363244 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1dc3bf-9536-4718-9178-b5a250d09089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:16.364317 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dc3bf-9536-4718-9178-b5a250d09089 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.364706 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dc3bf-9536-4718-9178-b5a250d09089 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.365083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dc3bf-9536-4718-9178-b5a250d09089 admin admin] PATH_INFO: `/v3/projects/347c30df00a24535baab8882fa994397/users/34995bbd20844db884cd44e0176cc9b2/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.378021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af69bd-c0d4-4e26-8810-104f5d18f522 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:16.384381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05db03-1658-445a-987c-4b4782c02878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:16.385329 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05db03-1658-445a-987c-4b4782c02878 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.385610 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05db03-1658-445a-987c-4b4782c02878 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.385886 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05db03-1658-445a-987c-4b4782c02878 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.386910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05db03-1658-445a-987c-4b4782c02878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:16.387146 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1dc3bf-9536-4718-9178-b5a250d09089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=347c30df00a24535baab8882fa994397, user_id=34995bbd20844db884cd44e0176cc9b2, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:16.388612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05db03-1658-445a-987c-4b4782c02878 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:16.390541 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1dc3bf-9536-4718-9178-b5a250d09089 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:16.405550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 946/1895] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:16.406392 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05db03-1658-445a-987c-4b4782c02878 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:16.417834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05db03-1658-445a-987c-4b4782c02878 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:16.423208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 950/1896] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:16 2022] PUT /identity/v3/projects/347c30df00a24535baab8882fa994397/users/34995bbd20844db884cd44e0176cc9b2/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:16.424220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1dc3bf-9536-4718-9178-b5a250d09089 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:16.507153 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56bf789-61f5-4921-aa09-48a92f350211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:16.508025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bf789-61f5-4921-aa09-48a92f350211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.508312 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bf789-61f5-4921-aa09-48a92f350211 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.509647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bf789-61f5-4921-aa09-48a92f350211 admin admin] PATH_INFO: `/v3/users/9ad267aebd3f44c981291a0d8305b865` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.522116 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56bf789-61f5-4921-aa09-48a92f350211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ad267aebd3f44c981291a0d8305b865)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:16.525412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56bf789-61f5-4921-aa09-48a92f350211 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:16.526741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1dc3bf-9536-4718-9178-b5a250d09089 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:16.549703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d327619-63bb-4c81-ae34-4a579e419252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:16.550954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d327619-63bb-4c81-ae34-4a579e419252 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.552135 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d327619-63bb-4c81-ae34-4a579e419252 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.552927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d327619-63bb-4c81-ae34-4a579e419252 admin admin] PATH_INFO: `/v3/projects/5be0cf2755284ed0b6016436593e2c71/users/8023f2252ff24e20bbb7c4ea4a89b24a/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.563845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d327619-63bb-4c81-ae34-4a579e419252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5be0cf2755284ed0b6016436593e2c71, user_id=8023f2252ff24e20bbb7c4ea4a89b24a, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:16.569059 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d327619-63bb-4c81-ae34-4a579e419252 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:16.594487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 951/1897] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:16 2022] PUT /identity/v3/projects/5be0cf2755284ed0b6016436593e2c71/users/8023f2252ff24e20bbb7c4ea4a89b24a/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:16.595396 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d327619-63bb-4c81-ae34-4a579e419252 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:16.657704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b56bf789-61f5-4921-aa09-48a92f350211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ad267aebd3f44c981291a0d8305b865'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:16.684817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b56bf789-61f5-4921-aa09-48a92f350211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ad267aebd3f44c981291a0d8305b865'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:16.687508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b56bf789-61f5-4921-aa09-48a92f350211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ad267aebd3f44c981291a0d8305b865'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:16.688381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b56bf789-61f5-4921-aa09-48a92f350211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ad267aebd3f44c981291a0d8305b865'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:16.719121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b56bf789-61f5-4921-aa09-48a92f350211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ad267aebd3f44c981291a0d8305b865'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:16.727554 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d327619-63bb-4c81-ae34-4a579e419252 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:16.733574 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5e4ca-dd71-494c-a803-1e7e8d6dbd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:16.735302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5e4ca-dd71-494c-a803-1e7e8d6dbd3b admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.735302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5e4ca-dd71-494c-a803-1e7e8d6dbd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.735302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5e4ca-dd71-494c-a803-1e7e8d6dbd3b admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.736676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5e4ca-dd71-494c-a803-1e7e8d6dbd3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:16.738320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5e4ca-dd71-494c-a803-1e7e8d6dbd3b admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:16.748950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 947/1898] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:16 2022] DELETE /identity/v3/users/9ad267aebd3f44c981291a0d8305b865 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:16.752890 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b01b5-afa0-40df-bdfb-1c5cdafa2312 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.753192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b01b5-afa0-40df-bdfb-1c5cdafa2312 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.753763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b01b5-afa0-40df-bdfb-1c5cdafa2312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.754569 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 952/1899] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:16.759676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f83ba-e793-488f-81de-39f9e558c22d None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.759977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f83ba-e793-488f-81de-39f9e558c22d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.760257 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f83ba-e793-488f-81de-39f9e558c22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.785157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-669f83ba-e793-488f-81de-39f9e558c22d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:16.787049 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-669f83ba-e793-488f-81de-39f9e558c22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:16.850690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-8a9b01b5-afa0-40df-bdfb-1c5cdafa2312 None None] MFA Rules not processed for user `8023f2252ff24e20bbb7c4ea4a89b24a`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:16.852050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8a9b01b5-afa0-40df-bdfb-1c5cdafa2312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:16.914921 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 953/1900] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:16.919480 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f83ba-e793-488f-81de-39f9e558c22d None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:16.926102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-669f83ba-e793-488f-81de-39f9e558c22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:16.939711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f83ba-e793-488f-81de-39f9e558c22d None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:16.944821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb240f6-daed-4fee-97a2-1b0d57e8bd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:16.945953 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb240f6-daed-4fee-97a2-1b0d57e8bd05 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:16.946341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb240f6-daed-4fee-97a2-1b0d57e8bd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:16.946732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb240f6-daed-4fee-97a2-1b0d57e8bd05 admin admin] PATH_INFO: `/v3/projects/347c30df00a24535baab8882fa994397/users/34995bbd20844db884cd44e0176cc9b2/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:16.948912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb240f6-daed-4fee-97a2-1b0d57e8bd05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=347c30df00a24535baab8882fa994397, user_id=34995bbd20844db884cd44e0176cc9b2, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:16.952464 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb240f6-daed-4fee-97a2-1b0d57e8bd05 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:16.964548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 954/1901] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:16 2022] PUT /identity/v3/projects/347c30df00a24535baab8882fa994397/users/34995bbd20844db884cd44e0176cc9b2/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:16.965515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb240f6-daed-4fee-97a2-1b0d57e8bd05 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:16.967998 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-dbb240f6-daed-4fee-97a2-1b0d57e8bd05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:17.001393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 948/1902] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:17.002788 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264aab9-449c-4dfa-8381-433c07fffde3 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:17.003156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264aab9-449c-4dfa-8381-433c07fffde3 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:17.003510 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264aab9-449c-4dfa-8381-433c07fffde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:17.051314 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5264aab9-449c-4dfa-8381-433c07fffde3 None None] MFA Rules not processed for user `34995bbd20844db884cd44e0176cc9b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:17.052632 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5264aab9-449c-4dfa-8381-433c07fffde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:17.066547 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb240f6-daed-4fee-97a2-1b0d57e8bd05 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:17.072685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3126622-7896-4937-ba42-20f074633b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:17.073941 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3126622-7896-4937-ba42-20f074633b39 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:17.074382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3126622-7896-4937-ba42-20f074633b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:17.075819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3126622-7896-4937-ba42-20f074633b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:17.131364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 949/1903] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:17.132134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264aab9-449c-4dfa-8381-433c07fffde3 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:17.133447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5264aab9-449c-4dfa-8381-433c07fffde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:17.152808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3126622-7896-4937-ba42-20f074633b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:17.154903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3126622-7896-4937-ba42-20f074633b39 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:17.164507 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 955/1904] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:17.165300 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3126622-7896-4937-ba42-20f074633b39 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:17.166650 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c3126622-7896-4937-ba42-20f074633b39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:17.200484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264aab9-449c-4dfa-8381-433c07fffde3 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:17.204654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8596e31-c21b-48aa-87d9-d17eb7f35da5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:17.205337 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8596e31-c21b-48aa-87d9-d17eb7f35da5 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:17.205597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8596e31-c21b-48aa-87d9-d17eb7f35da5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:17.205875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8596e31-c21b-48aa-87d9-d17eb7f35da5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:17.229182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3126622-7896-4937-ba42-20f074633b39 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:17.234536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed05bfa-9195-4832-8ae7-e41ee17a64a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:17.235251 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed05bfa-9195-4832-8ae7-e41ee17a64a4 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:17.235533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed05bfa-9195-4832-8ae7-e41ee17a64a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:17.235806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed05bfa-9195-4832-8ae7-e41ee17a64a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:17.246633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed05bfa-9195-4832-8ae7-e41ee17a64a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:17.250041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed05bfa-9195-4832-8ae7-e41ee17a64a4 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:17.260048 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 956/1905] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:17.282921 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8596e31-c21b-48aa-87d9-d17eb7f35da5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:17.284606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8596e31-c21b-48aa-87d9-d17eb7f35da5 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:17.302520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 950/1906] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:17.348642 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed05bfa-9195-4832-8ae7-e41ee17a64a4 service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:17.360215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed05bfa-9195-4832-8ae7-e41ee17a64a4 service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:17.364846 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb89aa-cb01-4371-ba0e-ebe11cf95725 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:17.365588 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb89aa-cb01-4371-ba0e-ebe11cf95725 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:17.366124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb89aa-cb01-4371-ba0e-ebe11cf95725 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:17.366411 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb89aa-cb01-4371-ba0e-ebe11cf95725 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:17.369179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-83fb89aa-cb01-4371-ba0e-ebe11cf95725 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:17.450935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb89aa-cb01-4371-ba0e-ebe11cf95725 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:17.456140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb89aa-cb01-4371-ba0e-ebe11cf95725 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:17.468041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 957/1907] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:52:17.564830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8596e31-c21b-48aa-87d9-d17eb7f35da5 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:17.566858 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c8596e31-c21b-48aa-87d9-d17eb7f35da5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:17.596270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8596e31-c21b-48aa-87d9-d17eb7f35da5 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:17.613630 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:17.618184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:17.618184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:17.618184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] PATH_INFO: `/v3/projects/fd2f8917ab214de99e83e179a1c9abcd` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:17.621374 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd2f8917ab214de99e83e179a1c9abcd)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:17.625513 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:17.704155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd2f8917ab214de99e83e179a1c9abcd', 'request_id': 'req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:17.735677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd2f8917ab214de99e83e179a1c9abcd', 'request_id': 'req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:17.758680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd2f8917ab214de99e83e179a1c9abcd', 'request_id': 'req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:17.774277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] The token cache is being invalidate because project fd2f8917ab214de99e83e179a1c9abcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:17.774569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:17.778845 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:17.780706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 951/1908] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:17 2022] DELETE /identity/v3/projects/fd2f8917ab214de99e83e179a1c9abcd => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:17.784568 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb89aa-cb01-4371-ba0e-ebe11cf95725 service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:17.799750 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb89aa-cb01-4371-ba0e-ebe11cf95725 service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:17.874935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bc327-3054-4b93-9553-281b3be6e6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:17.876087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bc327-3054-4b93-9553-281b3be6e6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:17.882309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bc327-3054-4b93-9553-281b3be6e6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:17.882309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bc327-3054-4b93-9553-281b3be6e6c9 admin admin] PATH_INFO: `/v3/users/2376cc2301824dcfa8e7550c7949f434` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:17.882309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995bc327-3054-4b93-9553-281b3be6e6c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2376cc2301824dcfa8e7550c7949f434)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:17.882309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995bc327-3054-4b93-9553-281b3be6e6c9 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:18.003236 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-995bc327-3054-4b93-9553-281b3be6e6c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2376cc2301824dcfa8e7550c7949f434'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.020928 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-995bc327-3054-4b93-9553-281b3be6e6c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2376cc2301824dcfa8e7550c7949f434'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.022216 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-995bc327-3054-4b93-9553-281b3be6e6c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2376cc2301824dcfa8e7550c7949f434'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.023052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-995bc327-3054-4b93-9553-281b3be6e6c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2376cc2301824dcfa8e7550c7949f434'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.031857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-995bc327-3054-4b93-9553-281b3be6e6c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2376cc2301824dcfa8e7550c7949f434'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.070885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 958/1909] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:17 2022] DELETE /identity/v3/users/2376cc2301824dcfa8e7550c7949f434 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:18.382573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:18.384093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:18.399806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be2a7d-f1bb-4aee-8a1a-dc754ef921cd admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:18.459528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bc327-3054-4b93-9553-281b3be6e6c9 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:18.461530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-995bc327-3054-4b93-9553-281b3be6e6c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:18.476149 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb5da8-a1bb-4a56-ae0b-4cf70f1173b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:18.477078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bc327-3054-4b93-9553-281b3be6e6c9 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:18.477317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb5da8-a1bb-4a56-ae0b-4cf70f1173b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:18.477605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb5da8-a1bb-4a56-ae0b-4cf70f1173b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:18.477883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb5da8-a1bb-4a56-ae0b-4cf70f1173b1 admin admin] PATH_INFO: `/v3/users/8c730524e557459db124c88b0e46e267` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:18.479362 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb5da8-a1bb-4a56-ae0b-4cf70f1173b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c730524e557459db124c88b0e46e267)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:18.482319 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb5da8-a1bb-4a56-ae0b-4cf70f1173b1 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:18.483314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e04abd-4462-4150-888c-282f5605b3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:18.484730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e04abd-4462-4150-888c-282f5605b3cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:18.485516 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e04abd-4462-4150-888c-282f5605b3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:18.485972 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e04abd-4462-4150-888c-282f5605b3cf admin admin] PATH_INFO: `/v3/projects/f6e750f9f47d4acc9f02050d068a3d7a` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:18.487444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e04abd-4462-4150-888c-282f5605b3cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6e750f9f47d4acc9f02050d068a3d7a)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:18.490502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e04abd-4462-4150-888c-282f5605b3cf admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:18.541446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-18cb5da8-a1bb-4a56-ae0b-4cf70f1173b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c730524e557459db124c88b0e46e267'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.549505 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-18e04abd-4462-4150-888c-282f5605b3cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6e750f9f47d4acc9f02050d068a3d7a', 'request_id': 'req-18e04abd-4462-4150-888c-282f5605b3cf'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.553733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-18cb5da8-a1bb-4a56-ae0b-4cf70f1173b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c730524e557459db124c88b0e46e267'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.554698 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-18cb5da8-a1bb-4a56-ae0b-4cf70f1173b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c730524e557459db124c88b0e46e267'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.555663 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-18cb5da8-a1bb-4a56-ae0b-4cf70f1173b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c730524e557459db124c88b0e46e267'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.557207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-18e04abd-4462-4150-888c-282f5605b3cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6e750f9f47d4acc9f02050d068a3d7a', 'request_id': 'req-18e04abd-4462-4150-888c-282f5605b3cf'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.563304 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-18cb5da8-a1bb-4a56-ae0b-4cf70f1173b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c730524e557459db124c88b0e46e267'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.565140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-18e04abd-4462-4150-888c-282f5605b3cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6e750f9f47d4acc9f02050d068a3d7a', 'request_id': 'req-18e04abd-4462-4150-888c-282f5605b3cf'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.576836 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-18e04abd-4462-4150-888c-282f5605b3cf admin admin] The token cache is being invalidate because project f6e750f9f47d4acc9f02050d068a3d7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:18.577346 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-18e04abd-4462-4150-888c-282f5605b3cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.578094 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 952/1910] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:18 2022] DELETE /identity/v3/users/8c730524e557459db124c88b0e46e267 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:18.578297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-18e04abd-4462-4150-888c-282f5605b3cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:18.579887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 959/1911] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:18 2022] DELETE /identity/v3/projects/f6e750f9f47d4acc9f02050d068a3d7a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:18.588415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e0d53-5dfd-4a30-96bb-54ec06c8bd32 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:18.589027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e0d53-5dfd-4a30-96bb-54ec06c8bd32 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:18.589354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e0d53-5dfd-4a30-96bb-54ec06c8bd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:18.589548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9e8be-4b05-49f3-a231-400a33990890 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:18.590004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9e8be-4b05-49f3-a231-400a33990890 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:18.590421 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9e8be-4b05-49f3-a231-400a33990890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:18.612436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ad6e0d53-5dfd-4a30-96bb-54ec06c8bd32 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:18.613744 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ad6e0d53-5dfd-4a30-96bb-54ec06c8bd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:18.617504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-c4c9e8be-4b05-49f3-a231-400a33990890 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:18.618965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c4c9e8be-4b05-49f3-a231-400a33990890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:18.694863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 953/1912] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:18.696578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 960/1913] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:18.704310 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69879a62-4361-4e0d-855f-b19e9884de26 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:18.704810 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69879a62-4361-4e0d-855f-b19e9884de26 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:18.705267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69879a62-4361-4e0d-855f-b19e9884de26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:18.706421 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9e8be-4b05-49f3-a231-400a33990890 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:18.707842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c4c9e8be-4b05-49f3-a231-400a33990890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:18.724164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-69879a62-4361-4e0d-855f-b19e9884de26 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:18.725850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-69879a62-4361-4e0d-855f-b19e9884de26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:18.734760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 954/1914] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:18.738729 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69879a62-4361-4e0d-855f-b19e9884de26 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:18.748372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69879a62-4361-4e0d-855f-b19e9884de26 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:18.753275 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a6dcd-f74a-4cda-9ff1-40a55306c04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:18.753993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a6dcd-f74a-4cda-9ff1-40a55306c04c admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:18.754343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a6dcd-f74a-4cda-9ff1-40a55306c04c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:18.754681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a6dcd-f74a-4cda-9ff1-40a55306c04c admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:18.755819 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a6dcd-f74a-4cda-9ff1-40a55306c04c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:18.755819 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a6dcd-f74a-4cda-9ff1-40a55306c04c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:18.757762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a6dcd-f74a-4cda-9ff1-40a55306c04c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:18.766641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9e8be-4b05-49f3-a231-400a33990890 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:18.770811 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 955/1915] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:18.771482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc9b25-a724-42fd-889c-8fba1fa8825e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:18.772186 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc9b25-a724-42fd-889c-8fba1fa8825e service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:18.772529 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc9b25-a724-42fd-889c-8fba1fa8825e service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:18.772862 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc9b25-a724-42fd-889c-8fba1fa8825e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:18.774823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a6dcd-f74a-4cda-9ff1-40a55306c04c admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:18.783832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc9b25-a724-42fd-889c-8fba1fa8825e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:18.785398 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a6dcd-f74a-4cda-9ff1-40a55306c04c admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:18.785529 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc9b25-a724-42fd-889c-8fba1fa8825e service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:18.790648 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c038f-8d56-45df-b646-53275b743220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:18.791415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c038f-8d56-45df-b646-53275b743220 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:18.791697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c038f-8d56-45df-b646-53275b743220 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:18.791973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c038f-8d56-45df-b646-53275b743220 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:18.792657 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c038f-8d56-45df-b646-53275b743220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:18.794586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c038f-8d56-45df-b646-53275b743220 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:18.795936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 961/1916] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:18.826453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 956/1917] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:18 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:18.829456 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc9b25-a724-42fd-889c-8fba1fa8825e service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:18.840934 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc9b25-a724-42fd-889c-8fba1fa8825e service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:18.915681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042def08-a578-4d82-b00d-0f1356ea5e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:18.916725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042def08-a578-4d82-b00d-0f1356ea5e04 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:18.917035 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042def08-a578-4d82-b00d-0f1356ea5e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:18.917312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042def08-a578-4d82-b00d-0f1356ea5e04 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:18.918133 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042def08-a578-4d82-b00d-0f1356ea5e04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:18.920132 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042def08-a578-4d82-b00d-0f1356ea5e04 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:18.947967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 962/1918] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:18 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:18.949349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c038f-8d56-45df-b646-53275b743220 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:18.961588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c038f-8d56-45df-b646-53275b743220 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:18.967248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e7baa-600e-4988-9493-9f6e906d18fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:18.968688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e7baa-600e-4988-9493-9f6e906d18fb admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:18.968688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e7baa-600e-4988-9493-9f6e906d18fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:18.968688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e7baa-600e-4988-9493-9f6e906d18fb admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:18.969873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e7baa-600e-4988-9493-9f6e906d18fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:18.971021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e7baa-600e-4988-9493-9f6e906d18fb admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:18.981888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 957/1919] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:18.986307 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042def08-a578-4d82-b00d-0f1356ea5e04 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:18.999169 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042def08-a578-4d82-b00d-0f1356ea5e04 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:19.005732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbb607-5cbb-40f6-9c66-0a846142b827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:19.006481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbb607-5cbb-40f6-9c66-0a846142b827 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:19.006873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbb607-5cbb-40f6-9c66-0a846142b827 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:19.007301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbb607-5cbb-40f6-9c66-0a846142b827 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:19.008265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cbb607-5cbb-40f6-9c66-0a846142b827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:19.009986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cbb607-5cbb-40f6-9c66-0a846142b827 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:19.027076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 963/1920] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:19.028365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e7baa-600e-4988-9493-9f6e906d18fb admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:19.043247 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e7baa-600e-4988-9493-9f6e906d18fb admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:19.049078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4710a7-0653-4d3f-8a9d-861efd6616a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:19.050167 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4710a7-0653-4d3f-8a9d-861efd6616a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:19.050448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4710a7-0653-4d3f-8a9d-861efd6616a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:19.050742 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4710a7-0653-4d3f-8a9d-861efd6616a0 admin admin] PATH_INFO: `/v3/projects/ee7802cd83ba42b498e048afec783616/users/9d6b50fedbd04218a7ecfb65c25b2165/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:19.108278 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4710a7-0653-4d3f-8a9d-861efd6616a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7802cd83ba42b498e048afec783616, user_id=9d6b50fedbd04218a7ecfb65c25b2165, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:19.112570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4710a7-0653-4d3f-8a9d-861efd6616a0 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:19.124015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbb607-5cbb-40f6-9c66-0a846142b827 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:19.126345 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-28cbb607-5cbb-40f6-9c66-0a846142b827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:19.134543 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 958/1921] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:19 2022] PUT /identity/v3/projects/ee7802cd83ba42b498e048afec783616/users/9d6b50fedbd04218a7ecfb65c25b2165/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:19.143705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbb607-5cbb-40f6-9c66-0a846142b827 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:19.145776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0251d7-25c6-4e4d-8c5b-d4c7e88102e1 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:19.146089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0251d7-25c6-4e4d-8c5b-d4c7e88102e1 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:19.146377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0251d7-25c6-4e4d-8c5b-d4c7e88102e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:19.192739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-3d0251d7-25c6-4e4d-8c5b-d4c7e88102e1 None None] MFA Rules not processed for user `9d6b50fedbd04218a7ecfb65c25b2165`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:19.194273 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-3d0251d7-25c6-4e4d-8c5b-d4c7e88102e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:19.302925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 959/1922] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:19.306431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0251d7-25c6-4e4d-8c5b-d4c7e88102e1 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:19.310644 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:19.311806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:19.312230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:19.312778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] PATH_INFO: `/v3/projects/76db9f5b2dea49f8b11294cd98fc11ce` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:19.316889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=76db9f5b2dea49f8b11294cd98fc11ce)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:19.319081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:19.321202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0251d7-25c6-4e4d-8c5b-d4c7e88102e1 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:19.327342 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5cae7-124e-4b65-b57f-9a8efe206a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:19.328281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5cae7-124e-4b65-b57f-9a8efe206a25 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:19.328648 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5cae7-124e-4b65-b57f-9a8efe206a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:19.329032 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5cae7-124e-4b65-b57f-9a8efe206a25 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:19.329854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5cae7-124e-4b65-b57f-9a8efe206a25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:19.332437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5cae7-124e-4b65-b57f-9a8efe206a25 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:19.367596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 960/1923] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:19 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:19.374280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5cae7-124e-4b65-b57f-9a8efe206a25 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:19.403622 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76db9f5b2dea49f8b11294cd98fc11ce', 'request_id': 'req-a6e86a7b-2990-4e8c-b3b0-168087718e6f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:19.411504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76db9f5b2dea49f8b11294cd98fc11ce', 'request_id': 'req-a6e86a7b-2990-4e8c-b3b0-168087718e6f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:19.424551 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76db9f5b2dea49f8b11294cd98fc11ce', 'request_id': 'req-a6e86a7b-2990-4e8c-b3b0-168087718e6f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:19.441418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] The token cache is being invalidate because project 76db9f5b2dea49f8b11294cd98fc11ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:19.441818 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:19.442613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:19.448832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 964/1924] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:19 2022] DELETE /identity/v3/projects/76db9f5b2dea49f8b11294cd98fc11ce => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:19.449724 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:19.452007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:19.515426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5cae7-124e-4b65-b57f-9a8efe206a25 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:19.563825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e86a7b-2990-4e8c-b3b0-168087718e6f admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:19.569136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc96d6c-cb86-4da7-80b8-ab2c69436f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:19.572907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc96d6c-cb86-4da7-80b8-ab2c69436f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:19.572907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc96d6c-cb86-4da7-80b8-ab2c69436f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:19.572907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc96d6c-cb86-4da7-80b8-ab2c69436f5a admin admin] PATH_INFO: `/v3/users/8023f2252ff24e20bbb7c4ea4a89b24a` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:19.572907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc96d6c-cb86-4da7-80b8-ab2c69436f5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8023f2252ff24e20bbb7c4ea4a89b24a)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:19.574317 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc96d6c-cb86-4da7-80b8-ab2c69436f5a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:19.595190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84425c-d8a4-411d-a3b6-7fea80860922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:19.595990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84425c-d8a4-411d-a3b6-7fea80860922 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:19.596418 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84425c-d8a4-411d-a3b6-7fea80860922 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:19.596707 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84425c-d8a4-411d-a3b6-7fea80860922 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:19.597409 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84425c-d8a4-411d-a3b6-7fea80860922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:19.599436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84425c-d8a4-411d-a3b6-7fea80860922 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:19.628382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 961/1925] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:19 2022] POST /identity/v3/users => generated 298 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:19.633035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84425c-d8a4-411d-a3b6-7fea80860922 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:19.635221 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ae84425c-d8a4-411d-a3b6-7fea80860922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:19.638487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0bc96d6c-cb86-4da7-80b8-ab2c69436f5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8023f2252ff24e20bbb7c4ea4a89b24a'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:19.650540 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84425c-d8a4-411d-a3b6-7fea80860922 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:19.652987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0bc96d6c-cb86-4da7-80b8-ab2c69436f5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8023f2252ff24e20bbb7c4ea4a89b24a'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:19.654348 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0bc96d6c-cb86-4da7-80b8-ab2c69436f5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8023f2252ff24e20bbb7c4ea4a89b24a'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:19.655199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0bc96d6c-cb86-4da7-80b8-ab2c69436f5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8023f2252ff24e20bbb7c4ea4a89b24a'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:19.658172 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f0e4c-15b6-4df5-af67-089d28528e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:19.659456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f0e4c-15b6-4df5-af67-089d28528e31 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:19.659812 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f0e4c-15b6-4df5-af67-089d28528e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:19.660095 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f0e4c-15b6-4df5-af67-089d28528e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:19.660833 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f0e4c-15b6-4df5-af67-089d28528e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:19.662509 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f0e4c-15b6-4df5-af67-089d28528e31 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:19.662896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0bc96d6c-cb86-4da7-80b8-ab2c69436f5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8023f2252ff24e20bbb7c4ea4a89b24a'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:19.682364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 962/1926] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:19.683590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f0e4c-15b6-4df5-af67-089d28528e31 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:19.687241 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-af1f0e4c-15b6-4df5-af67-089d28528e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:19.689286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 965/1927] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:19 2022] DELETE /identity/v3/users/8023f2252ff24e20bbb7c4ea4a89b24a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:19.705093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f0e4c-15b6-4df5-af67-089d28528e31 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:19.800219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f04acc-3589-4771-8653-5b5bcb7a3026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:19.801280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f04acc-3589-4771-8653-5b5bcb7a3026 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:19.801419 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f04acc-3589-4771-8653-5b5bcb7a3026 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:19.801977 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f04acc-3589-4771-8653-5b5bcb7a3026 admin admin] PATH_INFO: `/v3/projects/c51e59138ea14952948f02d43f8b8446/users/e389f2d624d248c59c9370693cf36ac4/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:19.834019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f04acc-3589-4771-8653-5b5bcb7a3026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c51e59138ea14952948f02d43f8b8446, user_id=e389f2d624d248c59c9370693cf36ac4, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:19.836586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f04acc-3589-4771-8653-5b5bcb7a3026 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:19.872036 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 963/1928] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:19 2022] PUT /identity/v3/projects/c51e59138ea14952948f02d43f8b8446/users/e389f2d624d248c59c9370693cf36ac4/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:19.873070 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc96d6c-cb86-4da7-80b8-ab2c69436f5a admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:19.886258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc96d6c-cb86-4da7-80b8-ab2c69436f5a admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:20.006432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f6f5c-9feb-4441-8973-707550feebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:20.007169 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f6f5c-9feb-4441-8973-707550feebe5 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:20.007461 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f6f5c-9feb-4441-8973-707550feebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:20.008341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f6f5c-9feb-4441-8973-707550feebe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:20.010488 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f6f5c-9feb-4441-8973-707550feebe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:20.016943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f6f5c-9feb-4441-8973-707550feebe5 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:20.041764 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 966/1929] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:52:20.043164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f04acc-3589-4771-8653-5b5bcb7a3026 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:20.061745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f04acc-3589-4771-8653-5b5bcb7a3026 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:20.077700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0825fb3-bcfe-4505-88af-f6c4b95fc662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:20.078625 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0825fb3-bcfe-4505-88af-f6c4b95fc662 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:20.078985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0825fb3-bcfe-4505-88af-f6c4b95fc662 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:20.079270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0825fb3-bcfe-4505-88af-f6c4b95fc662 admin admin] PATH_INFO: `/v3/projects/c51e59138ea14952948f02d43f8b8446/users/e389f2d624d248c59c9370693cf36ac4/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:20.086504 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0825fb3-bcfe-4505-88af-f6c4b95fc662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c51e59138ea14952948f02d43f8b8446, user_id=e389f2d624d248c59c9370693cf36ac4, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:20.089417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0825fb3-bcfe-4505-88af-f6c4b95fc662 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:20.118753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 964/1930] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:20 2022] PUT /identity/v3/projects/c51e59138ea14952948f02d43f8b8446/users/e389f2d624d248c59c9370693cf36ac4/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:20.124609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b212df-ce3d-4840-926e-39e9a9c81f00 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:20.125200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b212df-ce3d-4840-926e-39e9a9c81f00 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:20.126656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b212df-ce3d-4840-926e-39e9a9c81f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:20.174110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-c1b212df-ce3d-4840-926e-39e9a9c81f00 None None] MFA Rules not processed for user `e389f2d624d248c59c9370693cf36ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:20.175972 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c1b212df-ce3d-4840-926e-39e9a9c81f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:20.303585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0825fb3-bcfe-4505-88af-f6c4b95fc662 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:20.305290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d0825fb3-bcfe-4505-88af-f6c4b95fc662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:20.351901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 967/1931] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:20.369141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b212df-ce3d-4840-926e-39e9a9c81f00 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:20.372278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c1b212df-ce3d-4840-926e-39e9a9c81f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:20.419956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0825fb3-bcfe-4505-88af-f6c4b95fc662 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:20.425120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4b00e-86a5-4fe3-86af-37d80257fe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:20.425975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4b00e-86a5-4fe3-86af-37d80257fe50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:20.426264 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4b00e-86a5-4fe3-86af-37d80257fe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:20.426536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4b00e-86a5-4fe3-86af-37d80257fe50 admin admin] PATH_INFO: `/v3/projects/5be0cf2755284ed0b6016436593e2c71` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:20.427743 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4b00e-86a5-4fe3-86af-37d80257fe50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5be0cf2755284ed0b6016436593e2c71)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:20.430113 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4b00e-86a5-4fe3-86af-37d80257fe50 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:20.446695 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b212df-ce3d-4840-926e-39e9a9c81f00 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:20.451693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1e522-3fe7-425d-b7b6-c75d6bb44be5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:20.452490 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1e522-3fe7-425d-b7b6-c75d6bb44be5 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:20.452776 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1e522-3fe7-425d-b7b6-c75d6bb44be5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:20.453052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1e522-3fe7-425d-b7b6-c75d6bb44be5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:20.464848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1e522-3fe7-425d-b7b6-c75d6bb44be5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:20.466674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1e522-3fe7-425d-b7b6-c75d6bb44be5 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:20.479343 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 968/1932] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:20.489867 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-34e4b00e-86a5-4fe3-86af-37d80257fe50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5be0cf2755284ed0b6016436593e2c71', 'request_id': 'req-34e4b00e-86a5-4fe3-86af-37d80257fe50'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:20.497529 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-34e4b00e-86a5-4fe3-86af-37d80257fe50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5be0cf2755284ed0b6016436593e2c71', 'request_id': 'req-34e4b00e-86a5-4fe3-86af-37d80257fe50'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:20.508495 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-34e4b00e-86a5-4fe3-86af-37d80257fe50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5be0cf2755284ed0b6016436593e2c71', 'request_id': 'req-34e4b00e-86a5-4fe3-86af-37d80257fe50'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:20.521603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-34e4b00e-86a5-4fe3-86af-37d80257fe50 admin admin] The token cache is being invalidate because project 5be0cf2755284ed0b6016436593e2c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:20.522014 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-34e4b00e-86a5-4fe3-86af-37d80257fe50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:20.522928 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-34e4b00e-86a5-4fe3-86af-37d80257fe50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:20.525034 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 965/1933] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:20 2022] DELETE /identity/v3/projects/5be0cf2755284ed0b6016436593e2c71 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:20.530545 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df29d3c-c177-4147-810d-cd7c255cd031 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:20.530943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df29d3c-c177-4147-810d-cd7c255cd031 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:20.531303 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df29d3c-c177-4147-810d-cd7c255cd031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:20.556981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-4df29d3c-c177-4147-810d-cd7c255cd031 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:20.558496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-4df29d3c-c177-4147-810d-cd7c255cd031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:20.654852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 969/1934] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:20.660675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a7bf5-f9d6-4c4e-af99-2c03d312b3c0 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:20.661047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a7bf5-f9d6-4c4e-af99-2c03d312b3c0 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:20.661478 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a7bf5-f9d6-4c4e-af99-2c03d312b3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:20.685093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-843a7bf5-f9d6-4c4e-af99-2c03d312b3c0 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:20.686481 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-843a7bf5-f9d6-4c4e-af99-2c03d312b3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:20.696772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 966/1935] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:20.701893 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df29d3c-c177-4147-810d-cd7c255cd031 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:20.715330 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df29d3c-c177-4147-810d-cd7c255cd031 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:20.722263 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff8c43-ce4f-4f86-a22d-c2e85d52fdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:20.723130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff8c43-ce4f-4f86-a22d-c2e85d52fdaa admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:20.724167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff8c43-ce4f-4f86-a22d-c2e85d52fdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:20.724612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff8c43-ce4f-4f86-a22d-c2e85d52fdaa admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:20.725486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff8c43-ce4f-4f86-a22d-c2e85d52fdaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:20.726135 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff8c43-ce4f-4f86-a22d-c2e85d52fdaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:20.728227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff8c43-ce4f-4f86-a22d-c2e85d52fdaa admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:20.748880 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a7bf5-f9d6-4c4e-af99-2c03d312b3c0 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:20.755589 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 970/1936] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:20.760226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff8c43-ce4f-4f86-a22d-c2e85d52fdaa admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:20.761192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a7bf5-f9d6-4c4e-af99-2c03d312b3c0 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:20.764384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7cff8c43-ce4f-4f86-a22d-c2e85d52fdaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:20.769600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a17066-6918-4632-9f24-3b641fd9c566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:20.770621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a17066-6918-4632-9f24-3b641fd9c566 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:20.771273 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a17066-6918-4632-9f24-3b641fd9c566 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:20.771824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a17066-6918-4632-9f24-3b641fd9c566 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:20.773346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a17066-6918-4632-9f24-3b641fd9c566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:20.775706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a17066-6918-4632-9f24-3b641fd9c566 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:20.785668 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff8c43-ce4f-4f86-a22d-c2e85d52fdaa admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:20.790254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24413b3-7d49-48b5-a0f0-12f60ec221ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:20.790957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24413b3-7d49-48b5-a0f0-12f60ec221ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:20.791238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24413b3-7d49-48b5-a0f0-12f60ec221ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:20.791515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24413b3-7d49-48b5-a0f0-12f60ec221ab admin admin] PATH_INFO: `/v3/users/ca34b0a71d554031bf6b8439e0248795` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:20.792730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24413b3-7d49-48b5-a0f0-12f60ec221ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca34b0a71d554031bf6b8439e0248795)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:20.795593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24413b3-7d49-48b5-a0f0-12f60ec221ab admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:20.815073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 967/1937] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:20 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:20.816267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a17066-6918-4632-9f24-3b641fd9c566 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:20.819458 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f7a17066-6918-4632-9f24-3b641fd9c566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:20.858057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-e24413b3-7d49-48b5-a0f0-12f60ec221ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca34b0a71d554031bf6b8439e0248795'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:20.875943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-e24413b3-7d49-48b5-a0f0-12f60ec221ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca34b0a71d554031bf6b8439e0248795'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:20.876745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-e24413b3-7d49-48b5-a0f0-12f60ec221ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca34b0a71d554031bf6b8439e0248795'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:20.877488 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-e24413b3-7d49-48b5-a0f0-12f60ec221ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca34b0a71d554031bf6b8439e0248795'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:20.887182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-e24413b3-7d49-48b5-a0f0-12f60ec221ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca34b0a71d554031bf6b8439e0248795'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:20.905148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 971/1938] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:20 2022] DELETE /identity/v3/users/ca34b0a71d554031bf6b8439e0248795 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:20.906118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24413b3-7d49-48b5-a0f0-12f60ec221ab admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:20.908188 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e24413b3-7d49-48b5-a0f0-12f60ec221ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:20.914415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a17066-6918-4632-9f24-3b641fd9c566 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:20.992782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294575f-65f4-47b9-820d-e210740ab094 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:20.993570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294575f-65f4-47b9-820d-e210740ab094 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:20.993926 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294575f-65f4-47b9-820d-e210740ab094 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:20.994277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294575f-65f4-47b9-820d-e210740ab094 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:20.996322 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-1294575f-65f4-47b9-820d-e210740ab094 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:21.012681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24413b3-7d49-48b5-a0f0-12f60ec221ab admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:21.017801 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f290e1b-630b-495c-8813-aba6321273ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:21.018478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f290e1b-630b-495c-8813-aba6321273ec admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:21.018799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f290e1b-630b-495c-8813-aba6321273ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:21.019080 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f290e1b-630b-495c-8813-aba6321273ec admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:21.019859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f290e1b-630b-495c-8813-aba6321273ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:21.021909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f290e1b-630b-495c-8813-aba6321273ec admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:21.042860 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 972/1939] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:20 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:21.044994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b0ae3-3567-4d07-be83-945907d6dfd8 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:21.045412 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b0ae3-3567-4d07-be83-945907d6dfd8 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:21.045814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b0ae3-3567-4d07-be83-945907d6dfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:21.065144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-ec1b0ae3-3567-4d07-be83-945907d6dfd8 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:21.066359 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ec1b0ae3-3567-4d07-be83-945907d6dfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:21.067112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294575f-65f4-47b9-820d-e210740ab094 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:21.069207 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294575f-65f4-47b9-820d-e210740ab094 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:21.074428 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 973/1940] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:21.075127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b0ae3-3567-4d07-be83-945907d6dfd8 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:21.082896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 968/1941] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:21.089649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294575f-65f4-47b9-820d-e210740ab094 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:21.091361 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b0ae3-3567-4d07-be83-945907d6dfd8 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:21.091585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-1294575f-65f4-47b9-820d-e210740ab094 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:21.096878 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0639625-0c19-4a8d-9653-2f21a4a49240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:21.097652 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0639625-0c19-4a8d-9653-2f21a4a49240 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:21.097941 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0639625-0c19-4a8d-9653-2f21a4a49240 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:21.098273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0639625-0c19-4a8d-9653-2f21a4a49240 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:21.099924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0639625-0c19-4a8d-9653-2f21a4a49240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:21.101661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0639625-0c19-4a8d-9653-2f21a4a49240 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:21.121566 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 974/1942] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:21.122975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0639625-0c19-4a8d-9653-2f21a4a49240 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:21.138889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0639625-0c19-4a8d-9653-2f21a4a49240 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:21.146606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686fa54-9ff6-409f-80b3-456bccb1ce7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:21.147458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686fa54-9ff6-409f-80b3-456bccb1ce7f admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:21.147778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686fa54-9ff6-409f-80b3-456bccb1ce7f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:21.148061 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686fa54-9ff6-409f-80b3-456bccb1ce7f admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:21.148959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686fa54-9ff6-409f-80b3-456bccb1ce7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:21.151666 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686fa54-9ff6-409f-80b3-456bccb1ce7f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:21.180356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 975/1943] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:21.181402 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686fa54-9ff6-409f-80b3-456bccb1ce7f admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:21.193891 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686fa54-9ff6-409f-80b3-456bccb1ce7f admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:21.203582 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294575f-65f4-47b9-820d-e210740ab094 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:21.207528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d0fe0-c7ad-425d-9ba7-ed0cbd2d4c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:21.208085 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce2804-08db-4e16-9493-782c2e730bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:21.208436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d0fe0-c7ad-425d-9ba7-ed0cbd2d4c4f admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:21.209236 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce2804-08db-4e16-9493-782c2e730bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:21.209665 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce2804-08db-4e16-9493-782c2e730bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:21.210346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce2804-08db-4e16-9493-782c2e730bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:21.212440 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d0fe0-c7ad-425d-9ba7-ed0cbd2d4c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:21.213756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d0fe0-c7ad-425d-9ba7-ed0cbd2d4c4f admin admin] PATH_INFO: `/v3/projects/778e76fd939d4800a66a778a58eb2b1d/users/4ec646688c7049bea69f09252989ea83/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:21.241497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce2804-08db-4e16-9493-782c2e730bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:21.243335 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce2804-08db-4e16-9493-782c2e730bd3 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:21.248496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d0fe0-c7ad-425d-9ba7-ed0cbd2d4c4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=778e76fd939d4800a66a778a58eb2b1d, user_id=4ec646688c7049bea69f09252989ea83, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:21.253450 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d0fe0-c7ad-425d-9ba7-ed0cbd2d4c4f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:21.254502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 969/1944] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:52:21.284469 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 976/1945] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:21 2022] PUT /identity/v3/projects/778e76fd939d4800a66a778a58eb2b1d/users/4ec646688c7049bea69f09252989ea83/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:21.294600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3564b5-57db-43fa-820f-26a1858e60f6 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:21.296083 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3564b5-57db-43fa-820f-26a1858e60f6 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:21.296381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3564b5-57db-43fa-820f-26a1858e60f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:21.375364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-4a3564b5-57db-43fa-820f-26a1858e60f6 None None] MFA Rules not processed for user `4ec646688c7049bea69f09252989ea83`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:21.376583 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4a3564b5-57db-43fa-820f-26a1858e60f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:21.515770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 970/1946] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:21.520200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d0fe0-c7ad-425d-9ba7-ed0cbd2d4c4f admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:21.640356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d0fe0-c7ad-425d-9ba7-ed0cbd2d4c4f admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:21.647521 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17eb4f-8a70-4096-93d3-19a1a091e38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:21.648216 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17eb4f-8a70-4096-93d3-19a1a091e38f admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:21.648504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17eb4f-8a70-4096-93d3-19a1a091e38f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:21.648789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17eb4f-8a70-4096-93d3-19a1a091e38f admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:21.649610 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17eb4f-8a70-4096-93d3-19a1a091e38f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:21.652549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17eb4f-8a70-4096-93d3-19a1a091e38f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:21.711942 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 977/1947] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:21 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:21.713762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3564b5-57db-43fa-820f-26a1858e60f6 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:21.730246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3564b5-57db-43fa-820f-26a1858e60f6 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:21.813209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd338b0-bc07-4cb2-94ab-e451299cbe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:21.813891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd338b0-bc07-4cb2-94ab-e451299cbe84 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:21.814758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd338b0-bc07-4cb2-94ab-e451299cbe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:21.814758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd338b0-bc07-4cb2-94ab-e451299cbe84 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:21.815422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd338b0-bc07-4cb2-94ab-e451299cbe84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:21.821433 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd338b0-bc07-4cb2-94ab-e451299cbe84 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:21.853205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 971/1948] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:21 2022] POST /identity/v3/users => generated 298 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:21.854520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17eb4f-8a70-4096-93d3-19a1a091e38f admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:21.869197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17eb4f-8a70-4096-93d3-19a1a091e38f admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:21.877080 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239b631-48a3-4eed-b1a3-3b89fd7bf296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:21.878838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239b631-48a3-4eed-b1a3-3b89fd7bf296 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:21.879331 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239b631-48a3-4eed-b1a3-3b89fd7bf296 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:21.879837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239b631-48a3-4eed-b1a3-3b89fd7bf296 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:21.880774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239b631-48a3-4eed-b1a3-3b89fd7bf296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:21.884112 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239b631-48a3-4eed-b1a3-3b89fd7bf296 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:21.907233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 978/1949] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:52:21.908753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd338b0-bc07-4cb2-94ab-e451299cbe84 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:21.921595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd338b0-bc07-4cb2-94ab-e451299cbe84 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:21.929008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11e620-dc74-496d-88aa-aebbc35d810f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:21.931425 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11e620-dc74-496d-88aa-aebbc35d810f admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:21.932338 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11e620-dc74-496d-88aa-aebbc35d810f admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:21.932821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11e620-dc74-496d-88aa-aebbc35d810f admin admin] PATH_INFO: `/v3/projects/3e077452846045c5ac32662578662a0c/users/b9f7e96a32c54496b6efb0b5fd06926f/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:21.957422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11e620-dc74-496d-88aa-aebbc35d810f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e077452846045c5ac32662578662a0c, user_id=b9f7e96a32c54496b6efb0b5fd06926f, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:21.960440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11e620-dc74-496d-88aa-aebbc35d810f admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:21.987063 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 972/1950] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:21 2022] PUT /identity/v3/projects/3e077452846045c5ac32662578662a0c/users/b9f7e96a32c54496b6efb0b5fd06926f/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:21.991485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239b631-48a3-4eed-b1a3-3b89fd7bf296 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:22.009701 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11e620-dc74-496d-88aa-aebbc35d810f admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:22.017511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-fd11e620-dc74-496d-88aa-aebbc35d810f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:22.023241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239b631-48a3-4eed-b1a3-3b89fd7bf296 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:22.161142 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab507b11-c6cd-4a75-bdd2-4f1dfd305364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:22.161821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab507b11-c6cd-4a75-bdd2-4f1dfd305364 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:22.163979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab507b11-c6cd-4a75-bdd2-4f1dfd305364 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:22.164294 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab507b11-c6cd-4a75-bdd2-4f1dfd305364 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:22.165968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab507b11-c6cd-4a75-bdd2-4f1dfd305364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:22.167920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab507b11-c6cd-4a75-bdd2-4f1dfd305364 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:22.186543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 979/1951] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:22.187867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab507b11-c6cd-4a75-bdd2-4f1dfd305364 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:22.190285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11e620-dc74-496d-88aa-aebbc35d810f admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:22.203644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:22.203791 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab507b11-c6cd-4a75-bdd2-4f1dfd305364 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:22.204584 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:22.204984 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:22.205378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] PATH_INFO: `/v3/projects/41aa127145914e8a810abd619028e8c0` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:22.206943 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=41aa127145914e8a810abd619028e8c0)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:22.208717 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:22.210287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ba0da-fa3b-4bac-9afc-917fc3d4c674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:22.211411 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ba0da-fa3b-4bac-9afc-917fc3d4c674 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:22.211752 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ba0da-fa3b-4bac-9afc-917fc3d4c674 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:22.213293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ba0da-fa3b-4bac-9afc-917fc3d4c674 admin admin] PATH_INFO: `/v3/projects/3e077452846045c5ac32662578662a0c/users/b9f7e96a32c54496b6efb0b5fd06926f/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:22.216339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ba0da-fa3b-4bac-9afc-917fc3d4c674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e077452846045c5ac32662578662a0c, user_id=b9f7e96a32c54496b6efb0b5fd06926f, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:22.219870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ba0da-fa3b-4bac-9afc-917fc3d4c674 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:22.242988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 980/1952] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:22 2022] PUT /identity/v3/projects/3e077452846045c5ac32662578662a0c/users/b9f7e96a32c54496b6efb0b5fd06926f/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:22.244916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc2d97-75fe-4140-a6b6-4e74974b20a0 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:22.245332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc2d97-75fe-4140-a6b6-4e74974b20a0 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:22.245903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc2d97-75fe-4140-a6b6-4e74974b20a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:22.280738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41aa127145914e8a810abd619028e8c0', 'request_id': 'req-f429210b-6b06-4d1c-89de-426624dd5e0b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.289428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41aa127145914e8a810abd619028e8c0', 'request_id': 'req-f429210b-6b06-4d1c-89de-426624dd5e0b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.292784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-23bc2d97-75fe-4140-a6b6-4e74974b20a0 None None] MFA Rules not processed for user `b9f7e96a32c54496b6efb0b5fd06926f`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:22.296060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-23bc2d97-75fe-4140-a6b6-4e74974b20a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:22.303892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41aa127145914e8a810abd619028e8c0', 'request_id': 'req-f429210b-6b06-4d1c-89de-426624dd5e0b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.317074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] The token cache is being invalidate because project 41aa127145914e8a810abd619028e8c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:22.317393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.318012 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.319618 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 973/1953] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:22 2022] DELETE /identity/v3/projects/41aa127145914e8a810abd619028e8c0 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:22.320429 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:22.321709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:22.386936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 981/1954] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:22.387967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc2d97-75fe-4140-a6b6-4e74974b20a0 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:22.389578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-23bc2d97-75fe-4140-a6b6-4e74974b20a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:22.402913 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429210b-6b06-4d1c-89de-426624dd5e0b admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:22.403560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc2d97-75fe-4140-a6b6-4e74974b20a0 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:22.408678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade38213-3755-44a7-94a1-885f3540f3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:22.409673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade38213-3755-44a7-94a1-885f3540f3e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:22.410055 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade38213-3755-44a7-94a1-885f3540f3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:22.410371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0b8b0-7196-4754-b281-d35dfd7a4467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:22.410477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade38213-3755-44a7-94a1-885f3540f3e2 admin admin] PATH_INFO: `/v3/users/da643d51395a495494008c5b8a81d9d1` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:22.411822 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade38213-3755-44a7-94a1-885f3540f3e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da643d51395a495494008c5b8a81d9d1)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:22.411969 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0b8b0-7196-4754-b281-d35dfd7a4467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:22.412519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0b8b0-7196-4754-b281-d35dfd7a4467 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:22.412519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0b8b0-7196-4754-b281-d35dfd7a4467 admin admin] PATH_INFO: `/v3/users/ac8dbcfb37ee4548b1403442284df411` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:22.414120 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade38213-3755-44a7-94a1-885f3540f3e2 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:22.414494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0b8b0-7196-4754-b281-d35dfd7a4467 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac8dbcfb37ee4548b1403442284df411)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:22.417961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0b8b0-7196-4754-b281-d35dfd7a4467 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:22.486110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ade38213-3755-44a7-94a1-885f3540f3e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da643d51395a495494008c5b8a81d9d1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.499294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c5a0b8b0-7196-4754-b281-d35dfd7a4467 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac8dbcfb37ee4548b1403442284df411'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.509092 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ade38213-3755-44a7-94a1-885f3540f3e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da643d51395a495494008c5b8a81d9d1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.510031 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ade38213-3755-44a7-94a1-885f3540f3e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da643d51395a495494008c5b8a81d9d1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.511148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ade38213-3755-44a7-94a1-885f3540f3e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da643d51395a495494008c5b8a81d9d1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.518819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ade38213-3755-44a7-94a1-885f3540f3e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da643d51395a495494008c5b8a81d9d1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.519021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c5a0b8b0-7196-4754-b281-d35dfd7a4467 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac8dbcfb37ee4548b1403442284df411'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.519747 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c5a0b8b0-7196-4754-b281-d35dfd7a4467 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac8dbcfb37ee4548b1403442284df411'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.520377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c5a0b8b0-7196-4754-b281-d35dfd7a4467 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac8dbcfb37ee4548b1403442284df411'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.529768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c5a0b8b0-7196-4754-b281-d35dfd7a4467 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac8dbcfb37ee4548b1403442284df411'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:22.538022 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 982/1955] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:22 2022] DELETE /identity/v3/users/da643d51395a495494008c5b8a81d9d1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:22.539262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade38213-3755-44a7-94a1-885f3540f3e2 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:22.540719 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ade38213-3755-44a7-94a1-885f3540f3e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:22.552581 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 974/1956] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:22 2022] DELETE /identity/v3/users/ac8dbcfb37ee4548b1403442284df411 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:22.560728 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257c15e-ba5e-44fe-bc81-44ff8992a4d8 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:22.561070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257c15e-ba5e-44fe-bc81-44ff8992a4d8 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:22.561357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257c15e-ba5e-44fe-bc81-44ff8992a4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:22.584318 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-7257c15e-ba5e-44fe-bc81-44ff8992a4d8 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:22.585847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-7257c15e-ba5e-44fe-bc81-44ff8992a4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:22.616471 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade38213-3755-44a7-94a1-885f3540f3e2 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:22.666657 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 975/1957] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:22.680903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257c15e-ba5e-44fe-bc81-44ff8992a4d8 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:22.683187 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-7257c15e-ba5e-44fe-bc81-44ff8992a4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:22.708055 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b11b59-04eb-4270-91e7-393a20b23dd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:22.708631 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b11b59-04eb-4270-91e7-393a20b23dd5 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:22.708918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b11b59-04eb-4270-91e7-393a20b23dd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:22.709201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b11b59-04eb-4270-91e7-393a20b23dd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:22.712619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c0b11b59-04eb-4270-91e7-393a20b23dd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:22.787822 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b11b59-04eb-4270-91e7-393a20b23dd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:22.789974 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b11b59-04eb-4270-91e7-393a20b23dd5 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:22.803801 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 983/1958] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:22.827026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257c15e-ba5e-44fe-bc81-44ff8992a4d8 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:22.832315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112cbf4-eb59-46a7-85c6-e1c40aa347b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:22.833175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112cbf4-eb59-46a7-85c6-e1c40aa347b4 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:22.834254 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112cbf4-eb59-46a7-85c6-e1c40aa347b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:22.834730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112cbf4-eb59-46a7-85c6-e1c40aa347b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:22.846450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112cbf4-eb59-46a7-85c6-e1c40aa347b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:22.848547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112cbf4-eb59-46a7-85c6-e1c40aa347b4 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:22.861179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 976/1959] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:23.009343 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b11b59-04eb-4270-91e7-393a20b23dd5 service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:23.013918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c0b11b59-04eb-4270-91e7-393a20b23dd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:23.029830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112cbf4-eb59-46a7-85c6-e1c40aa347b4 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:23.031170 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-6112cbf4-eb59-46a7-85c6-e1c40aa347b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:23.057985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b11b59-04eb-4270-91e7-393a20b23dd5 service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:23.071409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe391d-9b1f-4b1d-9edc-71f9a9b6be78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:23.072088 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe391d-9b1f-4b1d-9edc-71f9a9b6be78 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:23.072377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe391d-9b1f-4b1d-9edc-71f9a9b6be78 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:23.072625 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112cbf4-eb59-46a7-85c6-e1c40aa347b4 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:23.072729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe391d-9b1f-4b1d-9edc-71f9a9b6be78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:23.075891 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-63fe391d-9b1f-4b1d-9edc-71f9a9b6be78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:23.092408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:23.093454 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:23.093739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:23.094888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] PATH_INFO: `/v3/projects/726370958a6e43c7aaca0fc3bf72759d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:23.096832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=726370958a6e43c7aaca0fc3bf72759d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:23.101701 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:23.182336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '726370958a6e43c7aaca0fc3bf72759d', 'request_id': 'req-970a761c-d365-4a3e-bd36-0ed15bd81f01'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:23.201706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '726370958a6e43c7aaca0fc3bf72759d', 'request_id': 'req-970a761c-d365-4a3e-bd36-0ed15bd81f01'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:23.204950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe391d-9b1f-4b1d-9edc-71f9a9b6be78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:23.206847 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe391d-9b1f-4b1d-9edc-71f9a9b6be78 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:23.209881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '726370958a6e43c7aaca0fc3bf72759d', 'request_id': 'req-970a761c-d365-4a3e-bd36-0ed15bd81f01'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:23.219122 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 984/1960] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:23.223963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] The token cache is being invalidate because project 726370958a6e43c7aaca0fc3bf72759d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:23.224395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:23.225535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:23.227657 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 977/1961] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:23 2022] DELETE /identity/v3/projects/726370958a6e43c7aaca0fc3bf72759d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:23.235076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b9acb-a754-485b-8cb2-34dc5494393e None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:23.235962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b9acb-a754-485b-8cb2-34dc5494393e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:23.236387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b9acb-a754-485b-8cb2-34dc5494393e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:23.271504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-ef6b9acb-a754-485b-8cb2-34dc5494393e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:23.271663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ef6b9acb-a754-485b-8cb2-34dc5494393e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:23.319786 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:23.321778 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:23.387222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 985/1962] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:23.395828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586be1e-166b-461f-971a-3a605225971b None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:23.397238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586be1e-166b-461f-971a-3a605225971b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:23.397551 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586be1e-166b-461f-971a-3a605225971b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:23.440612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7586be1e-166b-461f-971a-3a605225971b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:23.440612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7586be1e-166b-461f-971a-3a605225971b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:23.447772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a761c-d365-4a3e-bd36-0ed15bd81f01 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:23.457601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:23.459696 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:23.461844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 986/1963] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:23.462380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:23.462858 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] PATH_INFO: `/v3/projects/74f4bdc06a674d66bbe5f647e68afaa2` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:23.466874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586be1e-166b-461f-971a-3a605225971b None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:23.471731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=74f4bdc06a674d66bbe5f647e68afaa2)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:23.473837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:23.488134 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586be1e-166b-461f-971a-3a605225971b None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:23.503976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a7c56-beb5-49da-9558-42dbf1e8d01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:23.504950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a7c56-beb5-49da-9558-42dbf1e8d01c admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:23.505491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a7c56-beb5-49da-9558-42dbf1e8d01c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:23.506110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a7c56-beb5-49da-9558-42dbf1e8d01c admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:23.507022 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a7c56-beb5-49da-9558-42dbf1e8d01c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:23.507474 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a7c56-beb5-49da-9558-42dbf1e8d01c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:23.511062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a7c56-beb5-49da-9558-42dbf1e8d01c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:23.539955 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 987/1964] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:23.544679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a7c56-beb5-49da-9558-42dbf1e8d01c admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:23.581700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a7c56-beb5-49da-9558-42dbf1e8d01c admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:23.595871 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74f4bdc06a674d66bbe5f647e68afaa2', 'request_id': 'req-fa45f2b8-67a9-4a85-9345-99f6667656dc'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:23.607478 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74f4bdc06a674d66bbe5f647e68afaa2', 'request_id': 'req-fa45f2b8-67a9-4a85-9345-99f6667656dc'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:23.625114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74f4bdc06a674d66bbe5f647e68afaa2', 'request_id': 'req-fa45f2b8-67a9-4a85-9345-99f6667656dc'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:23.666909 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] The token cache is being invalidate because project 74f4bdc06a674d66bbe5f647e68afaa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:23.666909 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:23.670586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:23.674740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 978/1965] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:23 2022] DELETE /identity/v3/projects/74f4bdc06a674d66bbe5f647e68afaa2 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:23.678243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:23.680651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:23.716830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce274f6-c707-427b-bfa7-6d7cb9e89552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:23.717290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce274f6-c707-427b-bfa7-6d7cb9e89552 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:23.718067 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce274f6-c707-427b-bfa7-6d7cb9e89552 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:23.718859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce274f6-c707-427b-bfa7-6d7cb9e89552 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:23.719864 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce274f6-c707-427b-bfa7-6d7cb9e89552 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:23.725141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce274f6-c707-427b-bfa7-6d7cb9e89552 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:23.783693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 988/1966] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:23 2022] POST /identity/v3/projects => generated 383 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:23.785560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce274f6-c707-427b-bfa7-6d7cb9e89552 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:23.791894 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-dce274f6-c707-427b-bfa7-6d7cb9e89552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:23.840339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45f2b8-67a9-4a85-9345-99f6667656dc admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:23.922203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d1008-4c38-406f-aca7-675a4af289e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:23.923399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d1008-4c38-406f-aca7-675a4af289e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:23.923681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d1008-4c38-406f-aca7-675a4af289e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:23.923959 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d1008-4c38-406f-aca7-675a4af289e9 admin admin] PATH_INFO: `/v3/users/34995bbd20844db884cd44e0176cc9b2` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:23.925214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d1008-4c38-406f-aca7-675a4af289e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34995bbd20844db884cd44e0176cc9b2)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:23.926197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce274f6-c707-427b-bfa7-6d7cb9e89552 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:23.927178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d1008-4c38-406f-aca7-675a4af289e9 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:23.932167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475002d-a498-418e-95f8-2dba10b1da5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:23.932850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475002d-a498-418e-95f8-2dba10b1da5d admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:23.933691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475002d-a498-418e-95f8-2dba10b1da5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:23.933980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475002d-a498-418e-95f8-2dba10b1da5d admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:23.934744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475002d-a498-418e-95f8-2dba10b1da5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:23.937402 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475002d-a498-418e-95f8-2dba10b1da5d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:23.969737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 989/1967] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:23 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:23.970695 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475002d-a498-418e-95f8-2dba10b1da5d admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:23.973673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9475002d-a498-418e-95f8-2dba10b1da5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:23.988262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475002d-a498-418e-95f8-2dba10b1da5d admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:23.989556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0a6d1008-4c38-406f-aca7-675a4af289e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34995bbd20844db884cd44e0176cc9b2'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:23.995563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0456bb-0d7c-48c9-89e5-08bb5e1ad8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:23.996452 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0456bb-0d7c-48c9-89e5-08bb5e1ad8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:23.996774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0456bb-0d7c-48c9-89e5-08bb5e1ad8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:23.997998 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0456bb-0d7c-48c9-89e5-08bb5e1ad8dc admin admin] PATH_INFO: `/v3/users/4ec646688c7049bea69f09252989ea83` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:24.002175 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0456bb-0d7c-48c9-89e5-08bb5e1ad8dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ec646688c7049bea69f09252989ea83)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:24.005739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0456bb-0d7c-48c9-89e5-08bb5e1ad8dc admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:24.018463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0a6d1008-4c38-406f-aca7-675a4af289e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34995bbd20844db884cd44e0176cc9b2'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.019264 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0a6d1008-4c38-406f-aca7-675a4af289e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34995bbd20844db884cd44e0176cc9b2'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.019906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0a6d1008-4c38-406f-aca7-675a4af289e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34995bbd20844db884cd44e0176cc9b2'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.027557 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0a6d1008-4c38-406f-aca7-675a4af289e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34995bbd20844db884cd44e0176cc9b2'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.052272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 979/1968] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:23 2022] DELETE /identity/v3/users/34995bbd20844db884cd44e0176cc9b2 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:24.053305 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d1008-4c38-406f-aca7-675a4af289e9 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:24.055569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0a6d1008-4c38-406f-aca7-675a4af289e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.087310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-eb0456bb-0d7c-48c9-89e5-08bb5e1ad8dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ec646688c7049bea69f09252989ea83'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.111095 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-eb0456bb-0d7c-48c9-89e5-08bb5e1ad8dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ec646688c7049bea69f09252989ea83'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.113005 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-eb0456bb-0d7c-48c9-89e5-08bb5e1ad8dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ec646688c7049bea69f09252989ea83'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.114167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-eb0456bb-0d7c-48c9-89e5-08bb5e1ad8dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ec646688c7049bea69f09252989ea83'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.131384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-eb0456bb-0d7c-48c9-89e5-08bb5e1ad8dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ec646688c7049bea69f09252989ea83'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.154944 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 990/1969] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:23 2022] DELETE /identity/v3/users/4ec646688c7049bea69f09252989ea83 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:24.156206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0456bb-0d7c-48c9-89e5-08bb5e1ad8dc admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:24.158116 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-eb0456bb-0d7c-48c9-89e5-08bb5e1ad8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.174696 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d1008-4c38-406f-aca7-675a4af289e9 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:24.252419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0456bb-0d7c-48c9-89e5-08bb5e1ad8dc admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:24.258795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e8e51-e071-438b-878b-c127e66848b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:24.259694 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e8e51-e071-438b-878b-c127e66848b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:24.260218 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e8e51-e071-438b-878b-c127e66848b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:24.260497 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e8e51-e071-438b-878b-c127e66848b8 admin admin] PATH_INFO: `/v3/users/9d6b50fedbd04218a7ecfb65c25b2165` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:24.261801 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e8e51-e071-438b-878b-c127e66848b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d6b50fedbd04218a7ecfb65c25b2165)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:24.264257 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e8e51-e071-438b-878b-c127e66848b8 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:24.265588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302eaa5d-14e7-4cce-9d99-a712316d21ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:24.267439 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302eaa5d-14e7-4cce-9d99-a712316d21ad admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:24.268486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302eaa5d-14e7-4cce-9d99-a712316d21ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:24.268848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302eaa5d-14e7-4cce-9d99-a712316d21ad admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:24.269706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302eaa5d-14e7-4cce-9d99-a712316d21ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:24.271644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302eaa5d-14e7-4cce-9d99-a712316d21ad admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:24.288762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 980/1970] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:24.289818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18910b-1cd8-4e1f-9f32-83a526608698 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:24.290200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18910b-1cd8-4e1f-9f32-83a526608698 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:24.290556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18910b-1cd8-4e1f-9f32-83a526608698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:24.316995 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-8b18910b-1cd8-4e1f-9f32-83a526608698 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:24.318600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8b18910b-1cd8-4e1f-9f32-83a526608698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.331886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a70e8e51-e071-438b-878b-c127e66848b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d6b50fedbd04218a7ecfb65c25b2165'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.334433 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 981/1971] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:24.335690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18910b-1cd8-4e1f-9f32-83a526608698 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:24.337201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8b18910b-1cd8-4e1f-9f32-83a526608698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.345029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a70e8e51-e071-438b-878b-c127e66848b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d6b50fedbd04218a7ecfb65c25b2165'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.345912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a70e8e51-e071-438b-878b-c127e66848b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d6b50fedbd04218a7ecfb65c25b2165'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.347204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a70e8e51-e071-438b-878b-c127e66848b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d6b50fedbd04218a7ecfb65c25b2165'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.353219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18910b-1cd8-4e1f-9f32-83a526608698 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:24.358758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22389d84-494e-424b-a1ce-f66e53d91a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:24.359624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22389d84-494e-424b-a1ce-f66e53d91a27 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:24.359973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22389d84-494e-424b-a1ce-f66e53d91a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:24.360544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22389d84-494e-424b-a1ce-f66e53d91a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:24.360767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a70e8e51-e071-438b-878b-c127e66848b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d6b50fedbd04218a7ecfb65c25b2165'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.361406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22389d84-494e-424b-a1ce-f66e53d91a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:24.363448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22389d84-494e-424b-a1ce-f66e53d91a27 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:24.380659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 982/1972] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:24.381657 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22389d84-494e-424b-a1ce-f66e53d91a27 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:24.383641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-22389d84-494e-424b-a1ce-f66e53d91a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.385142 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 991/1973] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:24 2022] DELETE /identity/v3/users/9d6b50fedbd04218a7ecfb65c25b2165 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:24.386657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e8e51-e071-438b-878b-c127e66848b8 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:24.388867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a70e8e51-e071-438b-878b-c127e66848b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.451717 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22389d84-494e-424b-a1ce-f66e53d91a27 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:24.480455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e8e51-e071-438b-878b-c127e66848b8 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:24.486118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1c176-d691-470e-93c6-6a05d038203b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:24.487414 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1c176-d691-470e-93c6-6a05d038203b admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:24.487875 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1c176-d691-470e-93c6-6a05d038203b admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:24.488153 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1c176-d691-470e-93c6-6a05d038203b admin admin] PATH_INFO: `/v3/projects/e5c046789e39424cbba3c7e523af0c43/users/4ba98e87b9d4481aa222a61dc3c29cad/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:24.504383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1c176-d691-470e-93c6-6a05d038203b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5c046789e39424cbba3c7e523af0c43, user_id=4ba98e87b9d4481aa222a61dc3c29cad, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:24.507257 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1c176-d691-470e-93c6-6a05d038203b admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:24.514267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b5e33-8d71-454d-81e4-10fdc7eb06f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:24.515110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b5e33-8d71-454d-81e4-10fdc7eb06f6 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:24.515295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b5e33-8d71-454d-81e4-10fdc7eb06f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:24.515742 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b5e33-8d71-454d-81e4-10fdc7eb06f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:24.517903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-226b5e33-8d71-454d-81e4-10fdc7eb06f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.522771 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 992/1974] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:24 2022] PUT /identity/v3/projects/e5c046789e39424cbba3c7e523af0c43/users/4ba98e87b9d4481aa222a61dc3c29cad/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:24.530458 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b5e33-8d71-454d-81e4-10fdc7eb06f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:24.531941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b5e33-8d71-454d-81e4-10fdc7eb06f6 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:24.533599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833a245-c912-403e-be3b-3ee135ad7d25 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:24.533885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833a245-c912-403e-be3b-3ee135ad7d25 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:24.534157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833a245-c912-403e-be3b-3ee135ad7d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:24.544938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 983/1975] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:24.546164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b5e33-8d71-454d-81e4-10fdc7eb06f6 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:24.547728 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-226b5e33-8d71-454d-81e4-10fdc7eb06f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.552100 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e833a245-c912-403e-be3b-3ee135ad7d25 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:24.554101 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e833a245-c912-403e-be3b-3ee135ad7d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.560986 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b5e33-8d71-454d-81e4-10fdc7eb06f6 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:24.619081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 993/1976] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:24.621460 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849547f6-36f7-4343-93a5-492a3c6ff2c2 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:24.622610 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849547f6-36f7-4343-93a5-492a3c6ff2c2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:24.623435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849547f6-36f7-4343-93a5-492a3c6ff2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:24.626726 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:24.627715 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:24.628079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:24.628434 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] PATH_INFO: `/v3/projects/347c30df00a24535baab8882fa994397` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:24.629798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=347c30df00a24535baab8882fa994397)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:24.631915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:24.662083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-849547f6-36f7-4343-93a5-492a3c6ff2c2 None None] MFA Rules not processed for user `4ba98e87b9d4481aa222a61dc3c29cad`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:24.663604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-849547f6-36f7-4343-93a5-492a3c6ff2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.697496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '347c30df00a24535baab8882fa994397', 'request_id': 'req-84d2ab64-eba8-4064-88a0-aff5de61cb5e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.705334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '347c30df00a24535baab8882fa994397', 'request_id': 'req-84d2ab64-eba8-4064-88a0-aff5de61cb5e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.715929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '347c30df00a24535baab8882fa994397', 'request_id': 'req-84d2ab64-eba8-4064-88a0-aff5de61cb5e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.737399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] The token cache is being invalidate because project 347c30df00a24535baab8882fa994397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:24.737615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.738297 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:24.742832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 984/1977] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:24 2022] DELETE /identity/v3/projects/347c30df00a24535baab8882fa994397 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:24.742832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:24.745904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.757785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 994/1978] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:24.759002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf362c-0b2a-4152-838e-bce8a1163b08 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:24.759364 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf362c-0b2a-4152-838e-bce8a1163b08 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:24.759707 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf362c-0b2a-4152-838e-bce8a1163b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:24.785926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7daf362c-0b2a-4152-838e-bce8a1163b08 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:24.788520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7daf362c-0b2a-4152-838e-bce8a1163b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.810604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2ab64-eba8-4064-88a0-aff5de61cb5e admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:24.822738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323570-134d-483b-ab88-f2d73ba49dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:24.822738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323570-134d-483b-ab88-f2d73ba49dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:24.822738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323570-134d-483b-ab88-f2d73ba49dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:24.823064 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323570-134d-483b-ab88-f2d73ba49dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:24.824915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a8323570-134d-483b-ab88-f2d73ba49dd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.874160 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 995/1979] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:24.874989 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf362c-0b2a-4152-838e-bce8a1163b08 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:24.876340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7daf362c-0b2a-4152-838e-bce8a1163b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.891777 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf362c-0b2a-4152-838e-bce8a1163b08 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:24.897866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197b662-18b2-4bff-93ab-f6ce1e5c538c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:24.898558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197b662-18b2-4bff-93ab-f6ce1e5c538c admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:24.899164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197b662-18b2-4bff-93ab-f6ce1e5c538c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:24.899451 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197b662-18b2-4bff-93ab-f6ce1e5c538c admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:24.900372 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197b662-18b2-4bff-93ab-f6ce1e5c538c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:24.903601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197b662-18b2-4bff-93ab-f6ce1e5c538c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:24.927045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8323570-134d-483b-ab88-f2d73ba49dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:24.930117 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8323570-134d-483b-ab88-f2d73ba49dd5 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:24.947787 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 996/1980] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:24 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:24.947787 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c0763-1610-4194-a7cc-840a0bc4b046 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:24.947787 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c0763-1610-4194-a7cc-840a0bc4b046 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:24.947787 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c0763-1610-4194-a7cc-840a0bc4b046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:24.952709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 985/1981] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:24.953567 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323570-134d-483b-ab88-f2d73ba49dd5 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:24.954929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a8323570-134d-483b-ab88-f2d73ba49dd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:24.968287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323570-134d-483b-ab88-f2d73ba49dd5 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:24.968484 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-bf1c0763-1610-4194-a7cc-840a0bc4b046 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:24.969825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-bf1c0763-1610-4194-a7cc-840a0bc4b046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:25.035743 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 997/1982] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:25.038408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.039365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.039651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.039931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] PATH_INFO: `/v3/projects/778e76fd939d4800a66a778a58eb2b1d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.040111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c0763-1610-4194-a7cc-840a0bc4b046 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.041257 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=778e76fd939d4800a66a778a58eb2b1d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.043201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.050815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c0763-1610-4194-a7cc-840a0bc4b046 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.056714 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fef1f-d379-42fc-a7a2-e86f41bcf03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.057500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fef1f-d379-42fc-a7a2-e86f41bcf03e admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.057704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fef1f-d379-42fc-a7a2-e86f41bcf03e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.058388 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fef1f-d379-42fc-a7a2-e86f41bcf03e admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.058636 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fef1f-d379-42fc-a7a2-e86f41bcf03e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.060773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fef1f-d379-42fc-a7a2-e86f41bcf03e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.086764 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 998/1983] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:25 2022] POST /identity/v3/users => generated 299 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:25.087599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fef1f-d379-42fc-a7a2-e86f41bcf03e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.100836 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fef1f-d379-42fc-a7a2-e86f41bcf03e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.111874 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778e76fd939d4800a66a778a58eb2b1d', 'request_id': 'req-a6bb65a9-6120-438a-b572-a075eeb54ed6'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.118603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778e76fd939d4800a66a778a58eb2b1d', 'request_id': 'req-a6bb65a9-6120-438a-b572-a075eeb54ed6'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.127150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778e76fd939d4800a66a778a58eb2b1d', 'request_id': 'req-a6bb65a9-6120-438a-b572-a075eeb54ed6'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.138097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] The token cache is being invalidate because project 778e76fd939d4800a66a778a58eb2b1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:25.138391 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.139129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.140703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 986/1984] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:24 2022] DELETE /identity/v3/projects/778e76fd939d4800a66a778a58eb2b1d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:25.141613 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.143276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:25.206187 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc97caf-b085-4ec2-b4ce-48f522cedcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.207026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc97caf-b085-4ec2-b4ce-48f522cedcb1 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.207385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc97caf-b085-4ec2-b4ce-48f522cedcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.208264 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc97caf-b085-4ec2-b4ce-48f522cedcb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.209117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc97caf-b085-4ec2-b4ce-48f522cedcb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:25.209526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc97caf-b085-4ec2-b4ce-48f522cedcb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.211218 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc97caf-b085-4ec2-b4ce-48f522cedcb1 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.229560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 999/1985] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:25.229560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc97caf-b085-4ec2-b4ce-48f522cedcb1 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.229560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-6bc97caf-b085-4ec2-b4ce-48f522cedcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:25.234068 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb65a9-6120-438a-b572-a075eeb54ed6 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.239118 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f1fe6-97b6-48d0-aebe-333e5f56da29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.239930 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f1fe6-97b6-48d0-aebe-333e5f56da29 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.240218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f1fe6-97b6-48d0-aebe-333e5f56da29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.240495 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f1fe6-97b6-48d0-aebe-333e5f56da29 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.241484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f1fe6-97b6-48d0-aebe-333e5f56da29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.242478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc97caf-b085-4ec2-b4ce-48f522cedcb1 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.243118 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f1fe6-97b6-48d0-aebe-333e5f56da29 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.247376 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2307c77-bfd7-4c57-a492-adef337ddb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.248325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2307c77-bfd7-4c57-a492-adef337ddb57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.249320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2307c77-bfd7-4c57-a492-adef337ddb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.249687 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2307c77-bfd7-4c57-a492-adef337ddb57 admin admin] PATH_INFO: `/v3/users/b9f7e96a32c54496b6efb0b5fd06926f` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.251071 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2307c77-bfd7-4c57-a492-adef337ddb57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9f7e96a32c54496b6efb0b5fd06926f)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.253010 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2307c77-bfd7-4c57-a492-adef337ddb57 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.257497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 987/1986] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:25.257497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f1fe6-97b6-48d0-aebe-333e5f56da29 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.260584 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-885f1fe6-97b6-48d0-aebe-333e5f56da29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:25.275704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f1fe6-97b6-48d0-aebe-333e5f56da29 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.282034 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85853462-acb9-4ce9-b736-c43a31278d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.282895 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85853462-acb9-4ce9-b736-c43a31278d2b admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.283260 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85853462-acb9-4ce9-b736-c43a31278d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.283567 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85853462-acb9-4ce9-b736-c43a31278d2b admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.284380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85853462-acb9-4ce9-b736-c43a31278d2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.286915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85853462-acb9-4ce9-b736-c43a31278d2b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.309151 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-d2307c77-bfd7-4c57-a492-adef337ddb57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9f7e96a32c54496b6efb0b5fd06926f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.315862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 988/1987] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:25 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:25.316701 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85853462-acb9-4ce9-b736-c43a31278d2b admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.326025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-d2307c77-bfd7-4c57-a492-adef337ddb57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9f7e96a32c54496b6efb0b5fd06926f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.326808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-d2307c77-bfd7-4c57-a492-adef337ddb57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9f7e96a32c54496b6efb0b5fd06926f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.327494 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-d2307c77-bfd7-4c57-a492-adef337ddb57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9f7e96a32c54496b6efb0b5fd06926f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.334819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-d2307c77-bfd7-4c57-a492-adef337ddb57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9f7e96a32c54496b6efb0b5fd06926f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.358771 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1000/1988] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:25 2022] DELETE /identity/v3/users/b9f7e96a32c54496b6efb0b5fd06926f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:25.358771 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2307c77-bfd7-4c57-a492-adef337ddb57 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.362161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d2307c77-bfd7-4c57-a492-adef337ddb57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:25.392182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85853462-acb9-4ce9-b736-c43a31278d2b admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.439095 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2307c77-bfd7-4c57-a492-adef337ddb57 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.446273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb41341-1106-458c-afc6-e65c71408ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.446999 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb41341-1106-458c-afc6-e65c71408ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.447284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb41341-1106-458c-afc6-e65c71408ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.447563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb41341-1106-458c-afc6-e65c71408ee0 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.448258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb41341-1106-458c-afc6-e65c71408ee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.450810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb41341-1106-458c-afc6-e65c71408ee0 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.456345 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51a567-cd5d-442d-aedb-a7c008822173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.457241 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51a567-cd5d-442d-aedb-a7c008822173 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.458009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51a567-cd5d-442d-aedb-a7c008822173 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.458009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51a567-cd5d-442d-aedb-a7c008822173 admin admin] PATH_INFO: `/v3/projects/4850349064874e689dbff9e0ff497321/users/fe16fb4c480a405eb4b429fcb8b8e796/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.472893 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1001/1989] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:25 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:25.474063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb41341-1106-458c-afc6-e65c71408ee0 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.475960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-0cb41341-1106-458c-afc6-e65c71408ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:25.483894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51a567-cd5d-442d-aedb-a7c008822173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4850349064874e689dbff9e0ff497321, user_id=fe16fb4c480a405eb4b429fcb8b8e796, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.486621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51a567-cd5d-442d-aedb-a7c008822173 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.489781 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb41341-1106-458c-afc6-e65c71408ee0 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.498794 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.499668 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.499958 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.500237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] PATH_INFO: `/v3/projects/ee7802cd83ba42b498e048afec783616` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.501465 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 989/1990] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:25 2022] PUT /identity/v3/projects/4850349064874e689dbff9e0ff497321/users/fe16fb4c480a405eb4b429fcb8b8e796/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:25.502162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee7802cd83ba42b498e048afec783616)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.504289 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.504691 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51a567-cd5d-442d-aedb-a7c008822173 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.557446 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee7802cd83ba42b498e048afec783616', 'request_id': 'req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.564238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee7802cd83ba42b498e048afec783616', 'request_id': 'req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.572398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee7802cd83ba42b498e048afec783616', 'request_id': 'req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.573156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51a567-cd5d-442d-aedb-a7c008822173 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.583760 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] The token cache is being invalidate because project ee7802cd83ba42b498e048afec783616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:25.584264 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.584931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.586737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1002/1991] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:25 2022] DELETE /identity/v3/projects/ee7802cd83ba42b498e048afec783616 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:25.587304 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.589014 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:25.642005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e548db8-d608-4842-b0af-f54b9484e4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.642745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e548db8-d608-4842-b0af-f54b9484e4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.643024 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e548db8-d608-4842-b0af-f54b9484e4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.643305 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e548db8-d608-4842-b0af-f54b9484e4a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.644045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e548db8-d608-4842-b0af-f54b9484e4a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.645677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e548db8-d608-4842-b0af-f54b9484e4a5 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.656595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 990/1992] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:25.657534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e548db8-d608-4842-b0af-f54b9484e4a5 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.659234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8e548db8-d608-4842-b0af-f54b9484e4a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:25.665284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10666a-d08d-4a00-8d2d-cbb435fb7d40 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.671346 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c924d74-1bf4-404a-a5bf-787a1c06b75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.672031 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c924d74-1bf4-404a-a5bf-787a1c06b75d admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.672313 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c924d74-1bf4-404a-a5bf-787a1c06b75d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.672587 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c924d74-1bf4-404a-a5bf-787a1c06b75d admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.673312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c924d74-1bf4-404a-a5bf-787a1c06b75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.674184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e548db8-d608-4842-b0af-f54b9484e4a5 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.675059 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c924d74-1bf4-404a-a5bf-787a1c06b75d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.682927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074422b6-3fa4-46a9-9418-92dbb630e1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.684677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074422b6-3fa4-46a9-9418-92dbb630e1e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.685063 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074422b6-3fa4-46a9-9418-92dbb630e1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.686058 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1003/1993] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:25.686923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c924d74-1bf4-404a-a5bf-787a1c06b75d admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.687029 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074422b6-3fa4-46a9-9418-92dbb630e1e7 admin admin] PATH_INFO: `/v3/users/e389f2d624d248c59c9370693cf36ac4` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.689827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074422b6-3fa4-46a9-9418-92dbb630e1e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e389f2d624d248c59c9370693cf36ac4)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.690522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-6c924d74-1bf4-404a-a5bf-787a1c06b75d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:25.691844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074422b6-3fa4-46a9-9418-92dbb630e1e7 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.704420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c924d74-1bf4-404a-a5bf-787a1c06b75d admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.708977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d8df5-448d-4923-9e21-3099e7e0ed9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.709628 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d8df5-448d-4923-9e21-3099e7e0ed9c admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.709907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d8df5-448d-4923-9e21-3099e7e0ed9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.710184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d8df5-448d-4923-9e21-3099e7e0ed9c admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.710937 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d8df5-448d-4923-9e21-3099e7e0ed9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.712524 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d8df5-448d-4923-9e21-3099e7e0ed9c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.722627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1004/1994] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:25.723432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d8df5-448d-4923-9e21-3099e7e0ed9c admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.725080 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-932d8df5-448d-4923-9e21-3099e7e0ed9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:25.737975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d8df5-448d-4923-9e21-3099e7e0ed9c admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.741855 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-074422b6-3fa4-46a9-9418-92dbb630e1e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e389f2d624d248c59c9370693cf36ac4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.743795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.744406 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.744767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.745171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] PATH_INFO: `/v3/projects/3e077452846045c5ac32662578662a0c` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.747097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e077452846045c5ac32662578662a0c)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.749225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.755688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-074422b6-3fa4-46a9-9418-92dbb630e1e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e389f2d624d248c59c9370693cf36ac4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.756352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-074422b6-3fa4-46a9-9418-92dbb630e1e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e389f2d624d248c59c9370693cf36ac4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.756982 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-074422b6-3fa4-46a9-9418-92dbb630e1e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e389f2d624d248c59c9370693cf36ac4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.765209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-074422b6-3fa4-46a9-9418-92dbb630e1e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e389f2d624d248c59c9370693cf36ac4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.780987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 991/1995] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:25 2022] DELETE /identity/v3/users/e389f2d624d248c59c9370693cf36ac4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:25.782162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074422b6-3fa4-46a9-9418-92dbb630e1e7 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.791545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-074422b6-3fa4-46a9-9418-92dbb630e1e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:25.797815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e077452846045c5ac32662578662a0c', 'request_id': 'req-69ec5f46-6583-4f5e-94f7-0b0065a5b634'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.804607 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e077452846045c5ac32662578662a0c', 'request_id': 'req-69ec5f46-6583-4f5e-94f7-0b0065a5b634'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.812724 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e077452846045c5ac32662578662a0c', 'request_id': 'req-69ec5f46-6583-4f5e-94f7-0b0065a5b634'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.828609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] The token cache is being invalidate because project 3e077452846045c5ac32662578662a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:25.829120 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.829811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:25.832044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1005/1996] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:25 2022] DELETE /identity/v3/projects/3e077452846045c5ac32662578662a0c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:25.836065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:25.837923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:25.869164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074422b6-3fa4-46a9-9418-92dbb630e1e7 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.906230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec5f46-6583-4f5e-94f7-0b0065a5b634 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:25.911234 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded08a6-071e-455b-a820-985267878ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.912149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded08a6-071e-455b-a820-985267878ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.912433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded08a6-071e-455b-a820-985267878ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.912714 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded08a6-071e-455b-a820-985267878ae4 admin admin] PATH_INFO: `/v3/projects/0edcbbe2440f4666b68289635de4adea/users/2b6403f474fa468498fe0a5e753c1203/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.931306 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff2169-2912-4903-90e4-2c76e3865adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:25.931778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded08a6-071e-455b-a820-985267878ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0edcbbe2440f4666b68289635de4adea, user_id=2b6403f474fa468498fe0a5e753c1203, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.932248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2169-2912-4903-90e4-2c76e3865adc admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.932533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2169-2912-4903-90e4-2c76e3865adc admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.932812 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2169-2912-4903-90e4-2c76e3865adc admin admin] PATH_INFO: `/v3/projects/4850349064874e689dbff9e0ff497321/users/fe16fb4c480a405eb4b429fcb8b8e796/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.934033 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded08a6-071e-455b-a820-985267878ae4 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.935614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff2169-2912-4903-90e4-2c76e3865adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4850349064874e689dbff9e0ff497321, user_id=fe16fb4c480a405eb4b429fcb8b8e796, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:25.938442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff2169-2912-4903-90e4-2c76e3865adc admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:25.958162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1006/1998] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:25 2022] PUT /identity/v3/projects/0edcbbe2440f4666b68289635de4adea/users/2b6403f474fa468498fe0a5e753c1203/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:25.958162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b8d13-a22a-4c58-a8bb-2607a47190d9 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.958162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b8d13-a22a-4c58-a8bb-2607a47190d9 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.958162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b8d13-a22a-4c58-a8bb-2607a47190d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.961900 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 992/1998] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:25 2022] PUT /identity/v3/projects/4850349064874e689dbff9e0ff497321/users/fe16fb4c480a405eb4b429fcb8b8e796/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:25.963244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d4313-afeb-42d5-894f-5d0a302f373b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:25.963539 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d4313-afeb-42d5-894f-5d0a302f373b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:25.963821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d4313-afeb-42d5-894f-5d0a302f373b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:25.980505 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-576b8d13-a22a-4c58-a8bb-2607a47190d9 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:25.982447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-576b8d13-a22a-4c58-a8bb-2607a47190d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:25.988413 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-977d4313-afeb-42d5-894f-5d0a302f373b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:25.989651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-977d4313-afeb-42d5-894f-5d0a302f373b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:26.048901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1007/1999] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:26.051524 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e83451-94f5-4d45-8bf8-383d6d401d8a None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.052334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e83451-94f5-4d45-8bf8-383d6d401d8a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.052334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e83451-94f5-4d45-8bf8-383d6d401d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.053982 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 993/2000] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:26.056833 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb257f-ad11-424a-b676-e966f42189f3 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.057124 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb257f-ad11-424a-b676-e966f42189f3 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.057400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb257f-ad11-424a-b676-e966f42189f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.076801 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-b0cb257f-ad11-424a-b676-e966f42189f3 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:26.078256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b0cb257f-ad11-424a-b676-e966f42189f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:26.083655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-09e83451-94f5-4d45-8bf8-383d6d401d8a None None] MFA Rules not processed for user `fe16fb4c480a405eb4b429fcb8b8e796`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:26.087653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-09e83451-94f5-4d45-8bf8-383d6d401d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:26.091295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 994/2001] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:26.092172 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb257f-ad11-424a-b676-e966f42189f3 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:26.093553 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b0cb257f-ad11-424a-b676-e966f42189f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:26.159256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb257f-ad11-424a-b676-e966f42189f3 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:26.163088 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01368f51-12f5-4540-bac4-bd6050569c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:26.163926 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01368f51-12f5-4540-bac4-bd6050569c46 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.164224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01368f51-12f5-4540-bac4-bd6050569c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.164507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01368f51-12f5-4540-bac4-bd6050569c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.175097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01368f51-12f5-4540-bac4-bd6050569c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:26.177246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01368f51-12f5-4540-bac4-bd6050569c46 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:26.179771 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1008/2002] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:26.180468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e83451-94f5-4d45-8bf8-383d6d401d8a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:26.183927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-09e83451-94f5-4d45-8bf8-383d6d401d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:26.195581 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 995/2003] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:26.196838 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01368f51-12f5-4540-bac4-bd6050569c46 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:26.202207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e83451-94f5-4d45-8bf8-383d6d401d8a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:26.208132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01368f51-12f5-4540-bac4-bd6050569c46 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:26.208963 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:26.209840 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.210124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.210405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] PATH_INFO: `/v3/projects/c51e59138ea14952948f02d43f8b8446` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.211649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] RBAC: Authorizing `identity:delete_project(project_id=c51e59138ea14952948f02d43f8b8446)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:26.213108 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328e60a-82c7-4313-bbb8-81ebfd61f16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:26.213585 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:26.213761 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328e60a-82c7-4313-bbb8-81ebfd61f16a admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.214051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328e60a-82c7-4313-bbb8-81ebfd61f16a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.214612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328e60a-82c7-4313-bbb8-81ebfd61f16a admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.215591 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328e60a-82c7-4313-bbb8-81ebfd61f16a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:26.215904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328e60a-82c7-4313-bbb8-81ebfd61f16a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:26.217852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328e60a-82c7-4313-bbb8-81ebfd61f16a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:26.241398 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 996/2004] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:26.242341 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328e60a-82c7-4313-bbb8-81ebfd61f16a admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:26.245648 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-6328e60a-82c7-4313-bbb8-81ebfd61f16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:26.283418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c51e59138ea14952948f02d43f8b8446', 'request_id': 'req-29864152-4344-43c6-9094-b7c82768cece'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:26.295286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c51e59138ea14952948f02d43f8b8446', 'request_id': 'req-29864152-4344-43c6-9094-b7c82768cece'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:26.308961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c51e59138ea14952948f02d43f8b8446', 'request_id': 'req-29864152-4344-43c6-9094-b7c82768cece'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:26.321087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] The token cache is being invalidate because project c51e59138ea14952948f02d43f8b8446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:26.321461 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:26.322181 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:26.326912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1009/2005] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:26 2022] DELETE /identity/v3/projects/c51e59138ea14952948f02d43f8b8446 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:26.326912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:26.326912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:26.332861 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328e60a-82c7-4313-bbb8-81ebfd61f16a admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:26.396920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b8422-5833-4695-9914-4eeda716cfe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:26.397807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b8422-5833-4695-9914-4eeda716cfe9 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.398165 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b8422-5833-4695-9914-4eeda716cfe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.398657 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b8422-5833-4695-9914-4eeda716cfe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.401302 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-552b8422-5833-4695-9914-4eeda716cfe9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:26.404178 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29864152-4344-43c6-9094-b7c82768cece admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:26.409605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a7347-62c2-4beb-8f9e-38426b06d817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:26.410424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a7347-62c2-4beb-8f9e-38426b06d817 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.410721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a7347-62c2-4beb-8f9e-38426b06d817 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.411193 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a7347-62c2-4beb-8f9e-38426b06d817 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.411957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a7347-62c2-4beb-8f9e-38426b06d817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:26.413955 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a7347-62c2-4beb-8f9e-38426b06d817 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:26.441916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1010/2006] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:26 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:26.446337 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2589ba4-4ad9-40af-ae73-6c02c68b0508 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.446738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2589ba4-4ad9-40af-ae73-6c02c68b0508 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.447073 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2589ba4-4ad9-40af-ae73-6c02c68b0508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.467128 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b8422-5833-4695-9914-4eeda716cfe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:26.468770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b8422-5833-4695-9914-4eeda716cfe9 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:26.469515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d2589ba4-4ad9-40af-ae73-6c02c68b0508 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:26.470814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d2589ba4-4ad9-40af-ae73-6c02c68b0508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:26.480425 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 997/2007] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:26.482039 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b8422-5833-4695-9914-4eeda716cfe9 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:26.505642 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b8422-5833-4695-9914-4eeda716cfe9 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:26.570363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1011/2008] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:26.575527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c75af-9484-4657-b887-fd39d9749007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:26.575897 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c75af-9484-4657-b887-fd39d9749007 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.576445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c75af-9484-4657-b887-fd39d9749007 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.576734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c75af-9484-4657-b887-fd39d9749007 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.577480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c75af-9484-4657-b887-fd39d9749007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:26.577928 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad4fa8-3d2d-484a-bc8d-7486ec5f4048 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.578220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad4fa8-3d2d-484a-bc8d-7486ec5f4048 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.578620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad4fa8-3d2d-484a-bc8d-7486ec5f4048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.579443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c75af-9484-4657-b887-fd39d9749007 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:26.598868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a2ad4fa8-3d2d-484a-bc8d-7486ec5f4048 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:26.599882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a2ad4fa8-3d2d-484a-bc8d-7486ec5f4048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:26.602344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 998/2009] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:26 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:26.607151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c75af-9484-4657-b887-fd39d9749007 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:26.614003 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1012/2010] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:26.615066 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad4fa8-3d2d-484a-bc8d-7486ec5f4048 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:26.618467 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c75af-9484-4657-b887-fd39d9749007 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:26.624253 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f58b9-f9c0-49c9-b64e-f6725e7015c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:26.624881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f58b9-f9c0-49c9-b64e-f6725e7015c3 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.625351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f58b9-f9c0-49c9-b64e-f6725e7015c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.625485 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f58b9-f9c0-49c9-b64e-f6725e7015c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.627232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad4fa8-3d2d-484a-bc8d-7486ec5f4048 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:26.627343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f58b9-f9c0-49c9-b64e-f6725e7015c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:26.628354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f58b9-f9c0-49c9-b64e-f6725e7015c3 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:26.631851 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa39d7f-12ce-43f8-ba6f-7815de5276ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:26.632657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa39d7f-12ce-43f8-ba6f-7815de5276ec admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.633029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa39d7f-12ce-43f8-ba6f-7815de5276ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.633423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa39d7f-12ce-43f8-ba6f-7815de5276ec admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.634434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa39d7f-12ce-43f8-ba6f-7815de5276ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:26.634886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa39d7f-12ce-43f8-ba6f-7815de5276ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:26.636701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa39d7f-12ce-43f8-ba6f-7815de5276ec admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:26.643274 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 999/2011] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:26.645764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f58b9-f9c0-49c9-b64e-f6725e7015c3 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:26.653748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1013/2012] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:52:26.656162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa39d7f-12ce-43f8-ba6f-7815de5276ec admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:26.658949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f58b9-f9c0-49c9-b64e-f6725e7015c3 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:26.666582 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ac815-8f8e-424c-9fc6-479145de986f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:26.667290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ac815-8f8e-424c-9fc6-479145de986f admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.667583 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ac815-8f8e-424c-9fc6-479145de986f admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.668024 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ac815-8f8e-424c-9fc6-479145de986f admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.669317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ac815-8f8e-424c-9fc6-479145de986f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:26.669906 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa39d7f-12ce-43f8-ba6f-7815de5276ec admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:26.671251 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ac815-8f8e-424c-9fc6-479145de986f admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:26.677240 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85155667-b9a4-466d-a120-b653eff5adfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:26.682868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85155667-b9a4-466d-a120-b653eff5adfe admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.683264 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85155667-b9a4-466d-a120-b653eff5adfe admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.684297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85155667-b9a4-466d-a120-b653eff5adfe admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.685110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85155667-b9a4-466d-a120-b653eff5adfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:26.689774 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1000/2013] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:26.689908 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85155667-b9a4-466d-a120-b653eff5adfe admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:26.691741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ac815-8f8e-424c-9fc6-479145de986f admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:26.709690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ac815-8f8e-424c-9fc6-479145de986f admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:26.716873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb993156-0f09-46f3-aa29-fc49c97a1e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:26.718372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb993156-0f09-46f3-aa29-fc49c97a1e02 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.718746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb993156-0f09-46f3-aa29-fc49c97a1e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.719265 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb993156-0f09-46f3-aa29-fc49c97a1e02 admin admin] PATH_INFO: `/v3/projects/5eeafc3eb2794bad99d1b5b702ba1060/users/19ed4ceb5db74412adefa05b435bc9bf/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.724911 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1014/2014] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:26 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:26.725924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85155667-b9a4-466d-a120-b653eff5adfe admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:26.741059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb993156-0f09-46f3-aa29-fc49c97a1e02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eeafc3eb2794bad99d1b5b702ba1060, user_id=19ed4ceb5db74412adefa05b435bc9bf, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:26.744260 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb993156-0f09-46f3-aa29-fc49c97a1e02 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:26.759224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1001/2015] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:26 2022] PUT /identity/v3/projects/5eeafc3eb2794bad99d1b5b702ba1060/users/19ed4ceb5db74412adefa05b435bc9bf/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:26.770795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f246ae-3bce-4b48-9157-7cf4d3ff1c6b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.771210 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f246ae-3bce-4b48-9157-7cf4d3ff1c6b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.771574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f246ae-3bce-4b48-9157-7cf4d3ff1c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.801883 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85155667-b9a4-466d-a120-b653eff5adfe admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:26.803145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-94f246ae-3bce-4b48-9157-7cf4d3ff1c6b None None] MFA Rules not processed for user `19ed4ceb5db74412adefa05b435bc9bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:26.804522 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-94f246ae-3bce-4b48-9157-7cf4d3ff1c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:26.877423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0dcd8-e83a-4563-bbf7-c1f0a74ceeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:26.878251 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0dcd8-e83a-4563-bbf7-c1f0a74ceeff admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.878593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0dcd8-e83a-4563-bbf7-c1f0a74ceeff admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.878934 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0dcd8-e83a-4563-bbf7-c1f0a74ceeff admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.879632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0dcd8-e83a-4563-bbf7-c1f0a74ceeff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:26.881894 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0dcd8-e83a-4563-bbf7-c1f0a74ceeff admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:26.920452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1002/2016] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:26.922249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1015/2017] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:26 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:26.926287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f246ae-3bce-4b48-9157-7cf4d3ff1c6b None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:26.936654 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0dcd8-e83a-4563-bbf7-c1f0a74ceeff admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:26.942006 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-cfa0dcd8-e83a-4563-bbf7-c1f0a74ceeff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:26.947986 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f246ae-3bce-4b48-9157-7cf4d3ff1c6b None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:26.957868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c519eb5-c5eb-465f-9f6c-2862c40eb1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:26.958695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c519eb5-c5eb-465f-9f6c-2862c40eb1ed admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:26.958917 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c519eb5-c5eb-465f-9f6c-2862c40eb1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:26.959365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c519eb5-c5eb-465f-9f6c-2862c40eb1ed admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:26.960113 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c519eb5-c5eb-465f-9f6c-2862c40eb1ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:26.962605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c519eb5-c5eb-465f-9f6c-2862c40eb1ed admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:26.977431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1003/2018] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:52:26.981153 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c519eb5-c5eb-465f-9f6c-2862c40eb1ed admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.004046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c519eb5-c5eb-465f-9f6c-2862c40eb1ed admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.009650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f372d07-07a3-4b7b-a21c-d67be9b21b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.010410 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f372d07-07a3-4b7b-a21c-d67be9b21b40 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.010762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f372d07-07a3-4b7b-a21c-d67be9b21b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.011128 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f372d07-07a3-4b7b-a21c-d67be9b21b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.011969 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f372d07-07a3-4b7b-a21c-d67be9b21b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.014447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f372d07-07a3-4b7b-a21c-d67be9b21b40 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.030127 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1004/2019] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:27.034277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f372d07-07a3-4b7b-a21c-d67be9b21b40 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.048309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0dcd8-e83a-4563-bbf7-c1f0a74ceeff admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.052414 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eaef9c-5283-4bba-b227-ae4055137593 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.053235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaef9c-5283-4bba-b227-ae4055137593 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.053593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaef9c-5283-4bba-b227-ae4055137593 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.053947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaef9c-5283-4bba-b227-ae4055137593 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.065751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f372d07-07a3-4b7b-a21c-d67be9b21b40 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.067493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eaef9c-5283-4bba-b227-ae4055137593 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.069202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eaef9c-5283-4bba-b227-ae4055137593 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.071175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778303f-8962-42c4-8209-817fd0d50f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.072069 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778303f-8962-42c4-8209-817fd0d50f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.072421 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778303f-8962-42c4-8209-817fd0d50f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.072653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778303f-8962-42c4-8209-817fd0d50f14 admin admin] PATH_INFO: `/v3/projects/fece5ee41cda4016a523c3e3391271c8/users/0d2e8ab94896477b86b31483b68c8a38/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.081993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1016/2020] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:27.083826 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea070fa-a744-4a8d-9a9c-57e1e004376a None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.084137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea070fa-a744-4a8d-9a9c-57e1e004376a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.084516 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea070fa-a744-4a8d-9a9c-57e1e004376a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.090187 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778303f-8962-42c4-8209-817fd0d50f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fece5ee41cda4016a523c3e3391271c8, user_id=0d2e8ab94896477b86b31483b68c8a38, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.093078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778303f-8962-42c4-8209-817fd0d50f14 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.115457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-8ea070fa-a744-4a8d-9a9c-57e1e004376a None None] MFA Rules not processed for user `2b6403f474fa468498fe0a5e753c1203`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:27.116197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1005/2021] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:27 2022] PUT /identity/v3/projects/fece5ee41cda4016a523c3e3391271c8/users/0d2e8ab94896477b86b31483b68c8a38/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:27.126485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-8ea070fa-a744-4a8d-9a9c-57e1e004376a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:27.128101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6465af-7add-4428-a4ed-8dc7fa03fb53 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.128397 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6465af-7add-4428-a4ed-8dc7fa03fb53 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.128882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6465af-7add-4428-a4ed-8dc7fa03fb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.181104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-cf6465af-7add-4428-a4ed-8dc7fa03fb53 None None] MFA Rules not processed for user `0d2e8ab94896477b86b31483b68c8a38`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:27.182441 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-cf6465af-7add-4428-a4ed-8dc7fa03fb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:27.216499 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1017/2022] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:27.220989 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea070fa-a744-4a8d-9a9c-57e1e004376a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.225925 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-8ea070fa-a744-4a8d-9a9c-57e1e004376a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:27.301410 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1006/2023] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:27.305621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6465af-7add-4428-a4ed-8dc7fa03fb53 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.317379 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea070fa-a744-4a8d-9a9c-57e1e004376a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.318606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6465af-7add-4428-a4ed-8dc7fa03fb53 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.322497 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26eba8-fc69-4098-9a7f-dbb63566da19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.323203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26eba8-fc69-4098-9a7f-dbb63566da19 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.323485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26eba8-fc69-4098-9a7f-dbb63566da19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.323768 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26eba8-fc69-4098-9a7f-dbb63566da19 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.324458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26eba8-fc69-4098-9a7f-dbb63566da19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.324879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26332c-a680-4bd6-9ead-1f4235a9e9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.325601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26332c-a680-4bd6-9ead-1f4235a9e9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.325895 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26332c-a680-4bd6-9ead-1f4235a9e9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.326177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26332c-a680-4bd6-9ead-1f4235a9e9f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.326479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26eba8-fc69-4098-9a7f-dbb63566da19 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.326959 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26332c-a680-4bd6-9ead-1f4235a9e9f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.328884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26332c-a680-4bd6-9ead-1f4235a9e9f7 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.365244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1007/2024] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:27 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:27.366091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1018/2025] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:27 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:27.367685 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26332c-a680-4bd6-9ead-1f4235a9e9f7 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.369624 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26eba8-fc69-4098-9a7f-dbb63566da19 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.455706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26332c-a680-4bd6-9ead-1f4235a9e9f7 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.461195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4bbb1-e761-49ee-92ec-0d224358af3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.461933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4bbb1-e761-49ee-92ec-0d224358af3a admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.462343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4bbb1-e761-49ee-92ec-0d224358af3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.462740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4bbb1-e761-49ee-92ec-0d224358af3a admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.463500 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4bbb1-e761-49ee-92ec-0d224358af3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.466624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4bbb1-e761-49ee-92ec-0d224358af3a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.496652 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26eba8-fc69-4098-9a7f-dbb63566da19 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.501829 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1008/2026] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:27 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:27.502043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb4522-d62d-4a2e-a530-81cdcd464f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.502673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb4522-d62d-4a2e-a530-81cdcd464f43 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.502995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb4522-d62d-4a2e-a530-81cdcd464f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.503278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb4522-d62d-4a2e-a530-81cdcd464f43 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.503973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb4522-d62d-4a2e-a530-81cdcd464f43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.505772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4bbb1-e761-49ee-92ec-0d224358af3a admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.506203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb4522-d62d-4a2e-a530-81cdcd464f43 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.517701 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4bbb1-e761-49ee-92ec-0d224358af3a admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.529099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b93f85-9e14-4d2c-aa07-dc31a3b4eaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.529928 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b93f85-9e14-4d2c-aa07-dc31a3b4eaf9 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.530381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b93f85-9e14-4d2c-aa07-dc31a3b4eaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.536847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b93f85-9e14-4d2c-aa07-dc31a3b4eaf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.537826 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1019/2027] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:27 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:27.538080 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b93f85-9e14-4d2c-aa07-dc31a3b4eaf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.539029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb4522-d62d-4a2e-a530-81cdcd464f43 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.539882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b93f85-9e14-4d2c-aa07-dc31a3b4eaf9 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.551042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb4522-d62d-4a2e-a530-81cdcd464f43 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.551171 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1009/2028] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:27.555114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b93f85-9e14-4d2c-aa07-dc31a3b4eaf9 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.557588 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d185f-1929-4da3-84ca-2fce8a17022d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.558675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d185f-1929-4da3-84ca-2fce8a17022d admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.559206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d185f-1929-4da3-84ca-2fce8a17022d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.559626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d185f-1929-4da3-84ca-2fce8a17022d admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.560757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d185f-1929-4da3-84ca-2fce8a17022d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.562968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d185f-1929-4da3-84ca-2fce8a17022d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.564860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b93f85-9e14-4d2c-aa07-dc31a3b4eaf9 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.569589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa9178-38f2-4343-8b54-8c71e9cd0aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.570451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa9178-38f2-4343-8b54-8c71e9cd0aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.570761 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa9178-38f2-4343-8b54-8c71e9cd0aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.571040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa9178-38f2-4343-8b54-8c71e9cd0aeb admin admin] PATH_INFO: `/v3/projects/e464534e84de4491b568c6716574a689/users/1bbd1285beb34cd28e0b51f6ea059596/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.575633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1020/2029] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:27.579418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d185f-1929-4da3-84ca-2fce8a17022d admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.586172 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa9178-38f2-4343-8b54-8c71e9cd0aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e464534e84de4491b568c6716574a689, user_id=1bbd1285beb34cd28e0b51f6ea059596, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.588851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa9178-38f2-4343-8b54-8c71e9cd0aeb admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.589232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d185f-1929-4da3-84ca-2fce8a17022d admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.595097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c6c54-8442-4ad8-aa38-99f4d3768369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.595844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c6c54-8442-4ad8-aa38-99f4d3768369 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.596130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c6c54-8442-4ad8-aa38-99f4d3768369 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.596438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c6c54-8442-4ad8-aa38-99f4d3768369 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.597266 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c6c54-8442-4ad8-aa38-99f4d3768369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.598922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c6c54-8442-4ad8-aa38-99f4d3768369 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.602669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1010/2030] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:27 2022] PUT /identity/v3/projects/e464534e84de4491b568c6716574a689/users/1bbd1285beb34cd28e0b51f6ea059596/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:27.606689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa9178-38f2-4343-8b54-8c71e9cd0aeb admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.613376 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1021/2031] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:52:27.614518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c6c54-8442-4ad8-aa38-99f4d3768369 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.626011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c6c54-8442-4ad8-aa38-99f4d3768369 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.669339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa9178-38f2-4343-8b54-8c71e9cd0aeb admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.674232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd33c7-520d-4dc0-97b6-2384caabde3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.674927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd33c7-520d-4dc0-97b6-2384caabde3d admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.675214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd33c7-520d-4dc0-97b6-2384caabde3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.675492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd33c7-520d-4dc0-97b6-2384caabde3d admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.676212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd33c7-520d-4dc0-97b6-2384caabde3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.677805 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd33c7-520d-4dc0-97b6-2384caabde3d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.681180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b93c1-1577-4760-b00b-d87bb2725d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.682074 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b93c1-1577-4760-b00b-d87bb2725d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.682341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b93c1-1577-4760-b00b-d87bb2725d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.682622 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b93c1-1577-4760-b00b-d87bb2725d14 admin admin] PATH_INFO: `/v3/projects/351b4eba4d34476d8d295e41b393c060/users/8f08a56c67554a64bdc6fccf79c3733e/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.687622 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1011/2032] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:52:27.691792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd33c7-520d-4dc0-97b6-2384caabde3d admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.696829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b93c1-1577-4760-b00b-d87bb2725d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=351b4eba4d34476d8d295e41b393c060, user_id=8f08a56c67554a64bdc6fccf79c3733e, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.699551 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b93c1-1577-4760-b00b-d87bb2725d14 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.701548 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd33c7-520d-4dc0-97b6-2384caabde3d admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.706383 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9baf4-e03b-42ae-9a64-a5c86db996b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.707291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9baf4-e03b-42ae-9a64-a5c86db996b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.707591 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9baf4-e03b-42ae-9a64-a5c86db996b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.707850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9baf4-e03b-42ae-9a64-a5c86db996b7 admin admin] PATH_INFO: `/v3/projects/e464534e84de4491b568c6716574a689/users/1bbd1285beb34cd28e0b51f6ea059596/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.709833 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9baf4-e03b-42ae-9a64-a5c86db996b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e464534e84de4491b568c6716574a689, user_id=1bbd1285beb34cd28e0b51f6ea059596, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.712471 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9baf4-e03b-42ae-9a64-a5c86db996b7 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.712602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1022/2033] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:27 2022] PUT /identity/v3/projects/351b4eba4d34476d8d295e41b393c060/users/8f08a56c67554a64bdc6fccf79c3733e/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:27.723241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4f807-ff3e-4ecf-954e-6b8afb1a889f None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.726878 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1012/2034] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:27 2022] PUT /identity/v3/projects/e464534e84de4491b568c6716574a689/users/1bbd1285beb34cd28e0b51f6ea059596/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:27.727000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4f807-ff3e-4ecf-954e-6b8afb1a889f None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.727000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4f807-ff3e-4ecf-954e-6b8afb1a889f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.752323 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-9cb4f807-ff3e-4ecf-954e-6b8afb1a889f None None] MFA Rules not processed for user `8f08a56c67554a64bdc6fccf79c3733e`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:27.753620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9cb4f807-ff3e-4ecf-954e-6b8afb1a889f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:27.819778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1023/2035] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:27.824166 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9baf4-e03b-42ae-9a64-a5c86db996b7 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.896227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9baf4-e03b-42ae-9a64-a5c86db996b7 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:27.901814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8365d-c6cf-4a07-b1e6-209d3e74b24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:27.902394 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8365d-c6cf-4a07-b1e6-209d3e74b24d admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:27.903082 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8365d-c6cf-4a07-b1e6-209d3e74b24d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:27.903259 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8365d-c6cf-4a07-b1e6-209d3e74b24d admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:27.904535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8365d-c6cf-4a07-b1e6-209d3e74b24d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:27.906463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8365d-c6cf-4a07-b1e6-209d3e74b24d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:27.950435 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1013/2036] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:27 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:27.954312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4f807-ff3e-4ecf-954e-6b8afb1a889f None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:27.965843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4f807-ff3e-4ecf-954e-6b8afb1a889f None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:28.042825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6602d8-6d7e-4c2d-9094-625168dddfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:28.042825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6602d8-6d7e-4c2d-9094-625168dddfd9 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:28.042825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6602d8-6d7e-4c2d-9094-625168dddfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:28.042825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6602d8-6d7e-4c2d-9094-625168dddfd9 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:28.043423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6602d8-6d7e-4c2d-9094-625168dddfd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:28.043423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6602d8-6d7e-4c2d-9094-625168dddfd9 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:28.078789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1024/2037] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:27 2022] POST /identity/v3/users => generated 295 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:28.082881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8365d-c6cf-4a07-b1e6-209d3e74b24d admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:28.096931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8365d-c6cf-4a07-b1e6-209d3e74b24d admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:28.101580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a22ca-b0c8-43b2-b7ba-7e58dd420b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:28.102236 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a22ca-b0c8-43b2-b7ba-7e58dd420b67 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:28.102520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a22ca-b0c8-43b2-b7ba-7e58dd420b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:28.102823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a22ca-b0c8-43b2-b7ba-7e58dd420b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:28.103541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a22ca-b0c8-43b2-b7ba-7e58dd420b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:28.105136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a22ca-b0c8-43b2-b7ba-7e58dd420b67 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:28.115604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1014/2038] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:28.120226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6602d8-6d7e-4c2d-9094-625168dddfd9 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:28.132553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6602d8-6d7e-4c2d-9094-625168dddfd9 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:28.138977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab22dc2-a65d-430b-b46a-9cb32fbb04b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:28.140099 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab22dc2-a65d-430b-b46a-9cb32fbb04b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:28.140451 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab22dc2-a65d-430b-b46a-9cb32fbb04b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:28.140722 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab22dc2-a65d-430b-b46a-9cb32fbb04b1 admin admin] PATH_INFO: `/v3/projects/a39d3eece548497385d5c95bb15711a1/users/835225a651564330a16417ba79a30f81/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:28.162437 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab22dc2-a65d-430b-b46a-9cb32fbb04b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a39d3eece548497385d5c95bb15711a1, user_id=835225a651564330a16417ba79a30f81, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:28.165173 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab22dc2-a65d-430b-b46a-9cb32fbb04b1 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:28.179660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1025/2039] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:28 2022] PUT /identity/v3/projects/a39d3eece548497385d5c95bb15711a1/users/835225a651564330a16417ba79a30f81/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:28.184984 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a22ca-b0c8-43b2-b7ba-7e58dd420b67 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:28.196963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a22ca-b0c8-43b2-b7ba-7e58dd420b67 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:28.260383 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c83de-1788-4e9c-91f5-af98c34fb072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:28.261052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c83de-1788-4e9c-91f5-af98c34fb072 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:28.261335 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c83de-1788-4e9c-91f5-af98c34fb072 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:28.261610 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c83de-1788-4e9c-91f5-af98c34fb072 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:28.262386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c83de-1788-4e9c-91f5-af98c34fb072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:28.264162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c83de-1788-4e9c-91f5-af98c34fb072 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:28.277515 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1015/2040] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:28.278769 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab22dc2-a65d-430b-b46a-9cb32fbb04b1 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:28.290444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab22dc2-a65d-430b-b46a-9cb32fbb04b1 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:28.295764 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f609c-0633-4a04-a504-ba159669132c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:28.296641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f609c-0633-4a04-a504-ba159669132c admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:28.296916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f609c-0633-4a04-a504-ba159669132c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:28.297304 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f609c-0633-4a04-a504-ba159669132c admin admin] PATH_INFO: `/v3/projects/a39d3eece548497385d5c95bb15711a1/users/835225a651564330a16417ba79a30f81/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:28.300438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f609c-0633-4a04-a504-ba159669132c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a39d3eece548497385d5c95bb15711a1, user_id=835225a651564330a16417ba79a30f81, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:28.304042 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f609c-0633-4a04-a504-ba159669132c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:28.315627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1026/2041] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:28 2022] PUT /identity/v3/projects/a39d3eece548497385d5c95bb15711a1/users/835225a651564330a16417ba79a30f81/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:28.326660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808777a-8b74-4b39-8756-4ecd9d73fc97 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:28.327005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808777a-8b74-4b39-8756-4ecd9d73fc97 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:28.327289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808777a-8b74-4b39-8756-4ecd9d73fc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:28.359465 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5808777a-8b74-4b39-8756-4ecd9d73fc97 None None] MFA Rules not processed for user `835225a651564330a16417ba79a30f81`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:28.360708 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5808777a-8b74-4b39-8756-4ecd9d73fc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:28.395364 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c03fb-a870-44cf-83bb-0d2947defbb9 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:28.395660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c03fb-a870-44cf-83bb-0d2947defbb9 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:28.395943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c03fb-a870-44cf-83bb-0d2947defbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:28.434136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-457c03fb-a870-44cf-83bb-0d2947defbb9 None None] MFA Rules not processed for user `1bbd1285beb34cd28e0b51f6ea059596`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:28.436065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-457c03fb-a870-44cf-83bb-0d2947defbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:28.449245 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1016/2042] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:28.465423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808777a-8b74-4b39-8756-4ecd9d73fc97 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:28.531371 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1027/2043] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:28.540725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808777a-8b74-4b39-8756-4ecd9d73fc97 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:28.546395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6dd8b-75db-4148-96ac-3b53cc528e67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:28.547414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6dd8b-75db-4148-96ac-3b53cc528e67 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:28.547600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6dd8b-75db-4148-96ac-3b53cc528e67 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:28.551985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6dd8b-75db-4148-96ac-3b53cc528e67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:28.564812 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6dd8b-75db-4148-96ac-3b53cc528e67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:28.566778 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6dd8b-75db-4148-96ac-3b53cc528e67 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:28.569973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c03fb-a870-44cf-83bb-0d2947defbb9 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:28.571424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-457c03fb-a870-44cf-83bb-0d2947defbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:28.578183 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1017/2044] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:52:28.583731 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c03fb-a870-44cf-83bb-0d2947defbb9 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:28.587876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c77f0c-8087-4024-a897-9eaa3400ae03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:28.588468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c77f0c-8087-4024-a897-9eaa3400ae03 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:28.588748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c77f0c-8087-4024-a897-9eaa3400ae03 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:28.589021 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c77f0c-8087-4024-a897-9eaa3400ae03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:28.627807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6dd8b-75db-4148-96ac-3b53cc528e67 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:28.640305 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6dd8b-75db-4148-96ac-3b53cc528e67 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:28.644252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7c4aa-19df-445e-a2a0-a83e2f5b2364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:28.644901 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7c4aa-19df-445e-a2a0-a83e2f5b2364 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:28.645188 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7c4aa-19df-445e-a2a0-a83e2f5b2364 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:28.645468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7c4aa-19df-445e-a2a0-a83e2f5b2364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:28.661866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c77f0c-8087-4024-a897-9eaa3400ae03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:28.663370 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c77f0c-8087-4024-a897-9eaa3400ae03 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:28.674138 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1028/2045] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:52:28.715746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7c4aa-19df-445e-a2a0-a83e2f5b2364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:28.717251 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7c4aa-19df-445e-a2a0-a83e2f5b2364 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:28.735783 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1018/2046] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:52:32.136333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c77f0c-8087-4024-a897-9eaa3400ae03 service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:32.145845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c77f0c-8087-4024-a897-9eaa3400ae03 service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:32.150973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0d1d9-4c2f-4f14-a647-2ef82599ae85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:32.151623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0d1d9-4c2f-4f14-a647-2ef82599ae85 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:32.151901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0d1d9-4c2f-4f14-a647-2ef82599ae85 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:32.152174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0d1d9-4c2f-4f14-a647-2ef82599ae85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:32.164016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0d1d9-4c2f-4f14-a647-2ef82599ae85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:32.166349 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0d1d9-4c2f-4f14-a647-2ef82599ae85 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:32.176979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1029/2047] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:36.177021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7c4aa-19df-445e-a2a0-a83e2f5b2364 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:36.186799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7c4aa-19df-445e-a2a0-a83e2f5b2364 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:36.190882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2df11-3efb-4bcb-a84b-4f1847e65e11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:36.191617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2df11-3efb-4bcb-a84b-4f1847e65e11 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:36.191968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2df11-3efb-4bcb-a84b-4f1847e65e11 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:36.192312 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2df11-3efb-4bcb-a84b-4f1847e65e11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:36.194312 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c8d2df11-3efb-4bcb-a84b-4f1847e65e11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:36.258894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2df11-3efb-4bcb-a84b-4f1847e65e11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:36.260984 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2df11-3efb-4bcb-a84b-4f1847e65e11 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:36.271621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1019/2048] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:36.924987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0d1d9-4c2f-4f14-a647-2ef82599ae85 service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:36.993612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0d1d9-4c2f-4f14-a647-2ef82599ae85 service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:36.999073 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af312ec8-cf82-4a20-bac7-932fdac095dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:37.000243 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af312ec8-cf82-4a20-bac7-932fdac095dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:37.000523 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af312ec8-cf82-4a20-bac7-932fdac095dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:37.000806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af312ec8-cf82-4a20-bac7-932fdac095dd admin admin] PATH_INFO: `/v3/users/0d2e8ab94896477b86b31483b68c8a38` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:37.002074 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af312ec8-cf82-4a20-bac7-932fdac095dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d2e8ab94896477b86b31483b68c8a38)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:37.004033 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af312ec8-cf82-4a20-bac7-932fdac095dd admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:37.049676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-af312ec8-cf82-4a20-bac7-932fdac095dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d2e8ab94896477b86b31483b68c8a38'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.061698 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-af312ec8-cf82-4a20-bac7-932fdac095dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d2e8ab94896477b86b31483b68c8a38'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.062392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-af312ec8-cf82-4a20-bac7-932fdac095dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d2e8ab94896477b86b31483b68c8a38'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.063018 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-af312ec8-cf82-4a20-bac7-932fdac095dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d2e8ab94896477b86b31483b68c8a38'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.069780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-af312ec8-cf82-4a20-bac7-932fdac095dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d2e8ab94896477b86b31483b68c8a38'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.082435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1030/2049] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:36 2022] DELETE /identity/v3/users/0d2e8ab94896477b86b31483b68c8a38 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:37.087428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1fb9f-a118-4482-ad4d-508d613c194b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:37.087722 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1fb9f-a118-4482-ad4d-508d613c194b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:37.087999 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1fb9f-a118-4482-ad4d-508d613c194b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:37.104888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-72b1fb9f-a118-4482-ad4d-508d613c194b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:37.106284 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-72b1fb9f-a118-4482-ad4d-508d613c194b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:37.172759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1020/2050] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:37.182160 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af312ec8-cf82-4a20-bac7-932fdac095dd admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:37.183984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-af312ec8-cf82-4a20-bac7-932fdac095dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:37.257144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af312ec8-cf82-4a20-bac7-932fdac095dd admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:37.259119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e56ce4-cdb3-4f6e-9209-f8e9c176a00c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:37.259952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e56ce4-cdb3-4f6e-9209-f8e9c176a00c service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:37.260262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e56ce4-cdb3-4f6e-9209-f8e9c176a00c service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:37.260540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e56ce4-cdb3-4f6e-9209-f8e9c176a00c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:37.272789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e56ce4-cdb3-4f6e-9209-f8e9c176a00c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:37.274342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e56ce4-cdb3-4f6e-9209-f8e9c176a00c service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:37.288580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1031/2051] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:37.590487 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1fb9f-a118-4482-ad4d-508d613c194b None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:37.592633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-72b1fb9f-a118-4482-ad4d-508d613c194b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:37.604817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1fb9f-a118-4482-ad4d-508d613c194b None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:37.610413 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:37.611321 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:37.611610 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:37.611892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] PATH_INFO: `/v3/projects/fece5ee41cda4016a523c3e3391271c8` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:37.613253 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fece5ee41cda4016a523c3e3391271c8)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:37.615277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:37.672540 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fece5ee41cda4016a523c3e3391271c8', 'request_id': 'req-ccbc7f86-b7ec-420f-bede-16391a7c134c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.681750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fece5ee41cda4016a523c3e3391271c8', 'request_id': 'req-ccbc7f86-b7ec-420f-bede-16391a7c134c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.689820 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fece5ee41cda4016a523c3e3391271c8', 'request_id': 'req-ccbc7f86-b7ec-420f-bede-16391a7c134c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.702418 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] The token cache is being invalidate because project fece5ee41cda4016a523c3e3391271c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:37.702673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.703612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.705263 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1021/2052] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:37 2022] DELETE /identity/v3/projects/fece5ee41cda4016a523c3e3391271c8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:37.709408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e56ce4-cdb3-4f6e-9209-f8e9c176a00c service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:37.720976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e56ce4-cdb3-4f6e-9209-f8e9c176a00c service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:37.814022 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11187b8-8ba0-4f1e-9a73-bf0163db8566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:37.819744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11187b8-8ba0-4f1e-9a73-bf0163db8566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:37.820037 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11187b8-8ba0-4f1e-9a73-bf0163db8566 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:37.820335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11187b8-8ba0-4f1e-9a73-bf0163db8566 admin admin] PATH_INFO: `/v3/users/8f08a56c67554a64bdc6fccf79c3733e` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:37.821744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11187b8-8ba0-4f1e-9a73-bf0163db8566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f08a56c67554a64bdc6fccf79c3733e)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:37.823734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11187b8-8ba0-4f1e-9a73-bf0163db8566 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:37.878401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b11187b8-8ba0-4f1e-9a73-bf0163db8566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f08a56c67554a64bdc6fccf79c3733e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.901119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b11187b8-8ba0-4f1e-9a73-bf0163db8566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f08a56c67554a64bdc6fccf79c3733e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.902380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b11187b8-8ba0-4f1e-9a73-bf0163db8566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f08a56c67554a64bdc6fccf79c3733e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.903245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b11187b8-8ba0-4f1e-9a73-bf0163db8566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f08a56c67554a64bdc6fccf79c3733e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.913548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b11187b8-8ba0-4f1e-9a73-bf0163db8566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f08a56c67554a64bdc6fccf79c3733e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:37.928034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1032/2053] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:37 2022] DELETE /identity/v3/users/8f08a56c67554a64bdc6fccf79c3733e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:38.231471 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:38.233035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:38.258123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc7f86-b7ec-420f-bede-16391a7c134c admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:38.320867 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:38.321741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:38.322026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:38.322323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] PATH_INFO: `/v3/projects/351b4eba4d34476d8d295e41b393c060` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:38.323585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=351b4eba4d34476d8d295e41b393c060)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:38.325516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:38.376298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '351b4eba4d34476d8d295e41b393c060', 'request_id': 'req-e8738968-cb27-499b-80bb-0d4f88953d43'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:38.382657 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '351b4eba4d34476d8d295e41b393c060', 'request_id': 'req-e8738968-cb27-499b-80bb-0d4f88953d43'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:38.393250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '351b4eba4d34476d8d295e41b393c060', 'request_id': 'req-e8738968-cb27-499b-80bb-0d4f88953d43'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:38.406107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] The token cache is being invalidate because project 351b4eba4d34476d8d295e41b393c060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:38.406107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:38.406107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:38.408030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1022/2054] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:38 2022] DELETE /identity/v3/projects/351b4eba4d34476d8d295e41b393c060 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:38.411845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11187b8-8ba0-4f1e-9a73-bf0163db8566 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:38.414331 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b11187b8-8ba0-4f1e-9a73-bf0163db8566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:38.430571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11187b8-8ba0-4f1e-9a73-bf0163db8566 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:38.497080 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33bf74-87f6-404d-b699-767e794e069e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:38.498598 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33bf74-87f6-404d-b699-767e794e069e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:38.498967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33bf74-87f6-404d-b699-767e794e069e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:38.499244 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33bf74-87f6-404d-b699-767e794e069e admin admin] PATH_INFO: `/v3/users/835225a651564330a16417ba79a30f81` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:38.500665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33bf74-87f6-404d-b699-767e794e069e admin admin] RBAC: Authorizing `identity:delete_user(user_id=835225a651564330a16417ba79a30f81)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:38.503551 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33bf74-87f6-404d-b699-767e794e069e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:38.568252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3a33bf74-87f6-404d-b699-767e794e069e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '835225a651564330a16417ba79a30f81'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:38.583390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3a33bf74-87f6-404d-b699-767e794e069e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '835225a651564330a16417ba79a30f81'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:38.592590 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3a33bf74-87f6-404d-b699-767e794e069e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '835225a651564330a16417ba79a30f81'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:38.594007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3a33bf74-87f6-404d-b699-767e794e069e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '835225a651564330a16417ba79a30f81'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:38.602944 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3a33bf74-87f6-404d-b699-767e794e069e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '835225a651564330a16417ba79a30f81'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:38.621599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1033/2055] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:38 2022] DELETE /identity/v3/users/835225a651564330a16417ba79a30f81 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:38.851277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:38.854197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:38.871060 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8738968-cb27-499b-80bb-0d4f88953d43 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:38.954618 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dedf108-1812-403b-9334-fd4908c33a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:38.955934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedf108-1812-403b-9334-fd4908c33a59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:38.956228 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedf108-1812-403b-9334-fd4908c33a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:38.956663 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedf108-1812-403b-9334-fd4908c33a59 admin admin] PATH_INFO: `/v3/users/19ed4ceb5db74412adefa05b435bc9bf` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:38.958407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dedf108-1812-403b-9334-fd4908c33a59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19ed4ceb5db74412adefa05b435bc9bf)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:38.960721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dedf108-1812-403b-9334-fd4908c33a59 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:39.030579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33bf74-87f6-404d-b699-767e794e069e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:39.032730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3a33bf74-87f6-404d-b699-767e794e069e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:39.049446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-8dedf108-1812-403b-9334-fd4908c33a59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19ed4ceb5db74412adefa05b435bc9bf'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.056946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33bf74-87f6-404d-b699-767e794e069e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:39.064336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be001422-ee91-45f5-9aa3-dc930e817774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:39.065220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be001422-ee91-45f5-9aa3-dc930e817774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.065528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be001422-ee91-45f5-9aa3-dc930e817774 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.065805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be001422-ee91-45f5-9aa3-dc930e817774 admin admin] PATH_INFO: `/v3/projects/a39d3eece548497385d5c95bb15711a1` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:39.069785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be001422-ee91-45f5-9aa3-dc930e817774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a39d3eece548497385d5c95bb15711a1)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:39.071850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be001422-ee91-45f5-9aa3-dc930e817774 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:39.097488 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-8dedf108-1812-403b-9334-fd4908c33a59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19ed4ceb5db74412adefa05b435bc9bf'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.098558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-8dedf108-1812-403b-9334-fd4908c33a59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19ed4ceb5db74412adefa05b435bc9bf'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.099440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-8dedf108-1812-403b-9334-fd4908c33a59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19ed4ceb5db74412adefa05b435bc9bf'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.108118 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-8dedf108-1812-403b-9334-fd4908c33a59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19ed4ceb5db74412adefa05b435bc9bf'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.130325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1023/2056] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:38 2022] DELETE /identity/v3/users/19ed4ceb5db74412adefa05b435bc9bf => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:39.134566 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfba47-3eb9-4379-a8b7-6cf8cd87b8f5 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.135168 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfba47-3eb9-4379-a8b7-6cf8cd87b8f5 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.135892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfba47-3eb9-4379-a8b7-6cf8cd87b8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:39.143330 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-be001422-ee91-45f5-9aa3-dc930e817774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a39d3eece548497385d5c95bb15711a1', 'request_id': 'req-be001422-ee91-45f5-9aa3-dc930e817774'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.150500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-be001422-ee91-45f5-9aa3-dc930e817774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a39d3eece548497385d5c95bb15711a1', 'request_id': 'req-be001422-ee91-45f5-9aa3-dc930e817774'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.159939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-be001422-ee91-45f5-9aa3-dc930e817774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a39d3eece548497385d5c95bb15711a1', 'request_id': 'req-be001422-ee91-45f5-9aa3-dc930e817774'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.162671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-87bfba47-3eb9-4379-a8b7-6cf8cd87b8f5 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:39.163946 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-87bfba47-3eb9-4379-a8b7-6cf8cd87b8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:39.174194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-be001422-ee91-45f5-9aa3-dc930e817774 admin admin] The token cache is being invalidate because project a39d3eece548497385d5c95bb15711a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:39.174436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-be001422-ee91-45f5-9aa3-dc930e817774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.175367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-be001422-ee91-45f5-9aa3-dc930e817774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.177274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1034/2057] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:39 2022] DELETE /identity/v3/projects/a39d3eece548497385d5c95bb15711a1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:39.195171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d0836-bc33-4b10-8fff-ba525ff673ce None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.195952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d0836-bc33-4b10-8fff-ba525ff673ce None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.196454 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d0836-bc33-4b10-8fff-ba525ff673ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:39.225120 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-502d0836-bc33-4b10-8fff-ba525ff673ce None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:39.227144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-502d0836-bc33-4b10-8fff-ba525ff673ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:39.292914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1024/2058] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:39.310430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfba47-3eb9-4379-a8b7-6cf8cd87b8f5 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:39.315696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1035/2059] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:39.317352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b192e0a-df75-4350-aa21-b592270861c5 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.318638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-87bfba47-3eb9-4379-a8b7-6cf8cd87b8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:39.319627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b192e0a-df75-4350-aa21-b592270861c5 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.320159 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b192e0a-df75-4350-aa21-b592270861c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:39.344310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7b192e0a-df75-4350-aa21-b592270861c5 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:39.346277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7b192e0a-df75-4350-aa21-b592270861c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:39.357141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1036/2060] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:39.361453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b192e0a-df75-4350-aa21-b592270861c5 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:39.372058 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b192e0a-df75-4350-aa21-b592270861c5 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:39.378633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9c27c-7a37-4a60-be53-ef55df9c8ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:39.379333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9c27c-7a37-4a60-be53-ef55df9c8ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.379612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9c27c-7a37-4a60-be53-ef55df9c8ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.380189 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9c27c-7a37-4a60-be53-ef55df9c8ac7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:39.381093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9c27c-7a37-4a60-be53-ef55df9c8ac7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:39.381403 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9c27c-7a37-4a60-be53-ef55df9c8ac7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:39.383658 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9c27c-7a37-4a60-be53-ef55df9c8ac7 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:39.398745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1037/2061] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:39.402369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9c27c-7a37-4a60-be53-ef55df9c8ac7 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:39.413704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9c27c-7a37-4a60-be53-ef55df9c8ac7 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:39.418898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ff9b8-de89-4f79-ae62-9c283ce902d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:39.419716 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ff9b8-de89-4f79-ae62-9c283ce902d9 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.420078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ff9b8-de89-4f79-ae62-9c283ce902d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.420424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ff9b8-de89-4f79-ae62-9c283ce902d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:39.421183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ff9b8-de89-4f79-ae62-9c283ce902d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:39.423415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ff9b8-de89-4f79-ae62-9c283ce902d9 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:39.427570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfba47-3eb9-4379-a8b7-6cf8cd87b8f5 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:39.434412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff3f24-619b-4dd2-893f-5b6da92bf597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:39.435960 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff3f24-619b-4dd2-893f-5b6da92bf597 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.436248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff3f24-619b-4dd2-893f-5b6da92bf597 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.436529 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff3f24-619b-4dd2-893f-5b6da92bf597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:39.446697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff3f24-619b-4dd2-893f-5b6da92bf597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:39.448628 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff3f24-619b-4dd2-893f-5b6da92bf597 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:39.456377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1038/2062] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:39 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:39.460356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ff9b8-de89-4f79-ae62-9c283ce902d9 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:39.464551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1025/2063] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:39.537870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ff9b8-de89-4f79-ae62-9c283ce902d9 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:39.543726 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5c2d5-4be2-4dab-af4d-d26a29f49bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:39.544450 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5c2d5-4be2-4dab-af4d-d26a29f49bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.544817 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5c2d5-4be2-4dab-af4d-d26a29f49bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.545149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5c2d5-4be2-4dab-af4d-d26a29f49bd3 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:39.545931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c5c2d5-4be2-4dab-af4d-d26a29f49bd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:39.548147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c5c2d5-4be2-4dab-af4d-d26a29f49bd3 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:39.584993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1039/2064] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:39 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:39.589731 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff3f24-619b-4dd2-893f-5b6da92bf597 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:39.602259 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff3f24-619b-4dd2-893f-5b6da92bf597 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:39.606936 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef280f5-373b-4ef2-9958-fccb055a2b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:39.607773 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef280f5-373b-4ef2-9958-fccb055a2b72 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.608077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef280f5-373b-4ef2-9958-fccb055a2b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.608511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef280f5-373b-4ef2-9958-fccb055a2b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:39.609229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef280f5-373b-4ef2-9958-fccb055a2b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:39.611025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef280f5-373b-4ef2-9958-fccb055a2b72 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:39.629703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1026/2065] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:39.632461 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5c2d5-4be2-4dab-af4d-d26a29f49bd3 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:39.648526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5c2d5-4be2-4dab-af4d-d26a29f49bd3 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:39.653267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e99d7-75e4-4c3a-8284-11b2999085ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:39.653962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e99d7-75e4-4c3a-8284-11b2999085ee admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.654279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e99d7-75e4-4c3a-8284-11b2999085ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.654983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e99d7-75e4-4c3a-8284-11b2999085ee admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:39.655307 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e99d7-75e4-4c3a-8284-11b2999085ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:39.656912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e99d7-75e4-4c3a-8284-11b2999085ee admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:39.669277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1040/2066] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:39.673698 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef280f5-373b-4ef2-9958-fccb055a2b72 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:39.684834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef280f5-373b-4ef2-9958-fccb055a2b72 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:39.691081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f01d0-2d9e-4e4c-bc68-573c9e2d8d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:39.692160 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f01d0-2d9e-4e4c-bc68-573c9e2d8d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.692582 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f01d0-2d9e-4e4c-bc68-573c9e2d8d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.692867 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f01d0-2d9e-4e4c-bc68-573c9e2d8d65 admin admin] PATH_INFO: `/v3/projects/e2695816a7fb4bd7ab0ea420b156e6e6/users/c3a0de9ca12746be893b97d3cd11b826/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:39.712329 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f01d0-2d9e-4e4c-bc68-573c9e2d8d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2695816a7fb4bd7ab0ea420b156e6e6, user_id=c3a0de9ca12746be893b97d3cd11b826, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:39.715021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f01d0-2d9e-4e4c-bc68-573c9e2d8d65 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:39.730480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1027/2067] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:39 2022] PUT /identity/v3/projects/e2695816a7fb4bd7ab0ea420b156e6e6/users/c3a0de9ca12746be893b97d3cd11b826/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:39.735730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c50c3-f085-47ba-803c-1cea8014cc02 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.736231 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c50c3-f085-47ba-803c-1cea8014cc02 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.736665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c50c3-f085-47ba-803c-1cea8014cc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:39.765592 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-fb4c50c3-f085-47ba-803c-1cea8014cc02 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:39.767051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-fb4c50c3-f085-47ba-803c-1cea8014cc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:39.784466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f01d0-2d9e-4e4c-bc68-573c9e2d8d65 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:39.786595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d71f01d0-2d9e-4e4c-bc68-573c9e2d8d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:39.835647 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1041/2068] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:39.845591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c50c3-f085-47ba-803c-1cea8014cc02 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:39.847154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-fb4c50c3-f085-47ba-803c-1cea8014cc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:39.869002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f01d0-2d9e-4e4c-bc68-573c9e2d8d65 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:39.880472 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61deed7f-19df-4b25-b43a-9947db9c44de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:39.881411 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deed7f-19df-4b25-b43a-9947db9c44de admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.881699 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deed7f-19df-4b25-b43a-9947db9c44de admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.882487 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deed7f-19df-4b25-b43a-9947db9c44de admin admin] PATH_INFO: `/v3/projects/5eeafc3eb2794bad99d1b5b702ba1060` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:39.883497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61deed7f-19df-4b25-b43a-9947db9c44de admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eeafc3eb2794bad99d1b5b702ba1060)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:39.885535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61deed7f-19df-4b25-b43a-9947db9c44de admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:39.915549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c50c3-f085-47ba-803c-1cea8014cc02 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:39.919556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add068a4-16b8-4224-b78b-14ee5b75c0b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:39.920715 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add068a4-16b8-4224-b78b-14ee5b75c0b9 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.920999 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add068a4-16b8-4224-b78b-14ee5b75c0b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.921274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add068a4-16b8-4224-b78b-14ee5b75c0b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:39.930874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add068a4-16b8-4224-b78b-14ee5b75c0b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:39.932495 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add068a4-16b8-4224-b78b-14ee5b75c0b9 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:39.941182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-61deed7f-19df-4b25-b43a-9947db9c44de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eeafc3eb2794bad99d1b5b702ba1060', 'request_id': 'req-61deed7f-19df-4b25-b43a-9947db9c44de'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.947629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1042/2069] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:52:39.952709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-61deed7f-19df-4b25-b43a-9947db9c44de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eeafc3eb2794bad99d1b5b702ba1060', 'request_id': 'req-61deed7f-19df-4b25-b43a-9947db9c44de'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.962530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-61deed7f-19df-4b25-b43a-9947db9c44de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eeafc3eb2794bad99d1b5b702ba1060', 'request_id': 'req-61deed7f-19df-4b25-b43a-9947db9c44de'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.973687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-61deed7f-19df-4b25-b43a-9947db9c44de admin admin] The token cache is being invalidate because project 5eeafc3eb2794bad99d1b5b702ba1060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:39.974014 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-61deed7f-19df-4b25-b43a-9947db9c44de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.974718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-61deed7f-19df-4b25-b43a-9947db9c44de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:39.978767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1028/2070] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:39 2022] DELETE /identity/v3/projects/5eeafc3eb2794bad99d1b5b702ba1060 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:39.981196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab8134-2bda-4736-9b07-b0e4d2bda2ba None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:39.981768 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab8134-2bda-4736-9b07-b0e4d2bda2ba None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:39.982735 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab8134-2bda-4736-9b07-b0e4d2bda2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:40.003904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-2dab8134-2bda-4736-9b07-b0e4d2bda2ba None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:40.005088 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-2dab8134-2bda-4736-9b07-b0e4d2bda2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:40.076121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1043/2071] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:40.080137 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a31d3-346f-40b7-8cd4-b1465a6bd25e None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:40.080137 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a31d3-346f-40b7-8cd4-b1465a6bd25e None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:40.080137 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a31d3-346f-40b7-8cd4-b1465a6bd25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:40.098778 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-639a31d3-346f-40b7-8cd4-b1465a6bd25e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:40.100070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-639a31d3-346f-40b7-8cd4-b1465a6bd25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:40.109096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1029/2072] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:40.113420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab8134-2bda-4736-9b07-b0e4d2bda2ba None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:40.126665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab8134-2bda-4736-9b07-b0e4d2bda2ba None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:40.132578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f684c-ec0d-468f-8c62-5e00804fc96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:40.133253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f684c-ec0d-468f-8c62-5e00804fc96d admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:40.133533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f684c-ec0d-468f-8c62-5e00804fc96d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:40.133859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f684c-ec0d-468f-8c62-5e00804fc96d admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:40.134583 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f684c-ec0d-468f-8c62-5e00804fc96d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:40.135076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f684c-ec0d-468f-8c62-5e00804fc96d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:40.136910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f684c-ec0d-468f-8c62-5e00804fc96d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:40.164568 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1044/2073] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:52:40.168778 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a31d3-346f-40b7-8cd4-b1465a6bd25e None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:40.190040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a31d3-346f-40b7-8cd4-b1465a6bd25e None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:40.195696 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e08b55-0be4-49d1-bdcd-5e2c8cf5973b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:40.196626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e08b55-0be4-49d1-bdcd-5e2c8cf5973b admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:40.197280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e08b55-0be4-49d1-bdcd-5e2c8cf5973b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:40.197757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e08b55-0be4-49d1-bdcd-5e2c8cf5973b admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:40.198625 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e08b55-0be4-49d1-bdcd-5e2c8cf5973b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:40.200739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e08b55-0be4-49d1-bdcd-5e2c8cf5973b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:40.258076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1030/2074] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:40 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:40.258255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f684c-ec0d-468f-8c62-5e00804fc96d admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:40.283690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f684c-ec0d-468f-8c62-5e00804fc96d admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:40.378420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464c038-4fab-4b41-aa9e-702df5e41155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:40.379265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464c038-4fab-4b41-aa9e-702df5e41155 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:40.379618 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464c038-4fab-4b41-aa9e-702df5e41155 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:40.380200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464c038-4fab-4b41-aa9e-702df5e41155 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:40.381419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464c038-4fab-4b41-aa9e-702df5e41155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:40.384429 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464c038-4fab-4b41-aa9e-702df5e41155 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:40.413879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1045/2075] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:40 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:40.418082 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e08b55-0be4-49d1-bdcd-5e2c8cf5973b admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:40.434942 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e08b55-0be4-49d1-bdcd-5e2c8cf5973b admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:40.451385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca74d7f-f34f-4e35-9ed2-96494eb9ba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:40.453191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca74d7f-f34f-4e35-9ed2-96494eb9ba51 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:40.453482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca74d7f-f34f-4e35-9ed2-96494eb9ba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:40.455198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca74d7f-f34f-4e35-9ed2-96494eb9ba51 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:40.456700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca74d7f-f34f-4e35-9ed2-96494eb9ba51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:40.461399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca74d7f-f34f-4e35-9ed2-96494eb9ba51 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:40.497214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1031/2076] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:52:40.498350 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464c038-4fab-4b41-aa9e-702df5e41155 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:40.510977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464c038-4fab-4b41-aa9e-702df5e41155 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:40.516673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc577854-8fee-45a9-8081-28ac8b30be34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:40.517355 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc577854-8fee-45a9-8081-28ac8b30be34 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:40.517805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc577854-8fee-45a9-8081-28ac8b30be34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:40.518181 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc577854-8fee-45a9-8081-28ac8b30be34 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:40.519040 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc577854-8fee-45a9-8081-28ac8b30be34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:40.521176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc577854-8fee-45a9-8081-28ac8b30be34 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:40.537029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1046/2077] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:52:40.538448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca74d7f-f34f-4e35-9ed2-96494eb9ba51 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:40.553216 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca74d7f-f34f-4e35-9ed2-96494eb9ba51 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:40.559876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778e3da-90e2-47d9-836d-99e9329a824e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:40.562896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778e3da-90e2-47d9-836d-99e9329a824e admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:40.562896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778e3da-90e2-47d9-836d-99e9329a824e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:40.562896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778e3da-90e2-47d9-836d-99e9329a824e admin admin] PATH_INFO: `/v3/projects/1aef84932f4f4725bbc4095f07ebf56e/users/004034ab6aca41ae893214bf0e81841b/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:40.580778 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778e3da-90e2-47d9-836d-99e9329a824e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aef84932f4f4725bbc4095f07ebf56e, user_id=004034ab6aca41ae893214bf0e81841b, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:40.583928 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778e3da-90e2-47d9-836d-99e9329a824e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:40.605756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1032/2078] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:40 2022] PUT /identity/v3/projects/1aef84932f4f4725bbc4095f07ebf56e/users/004034ab6aca41ae893214bf0e81841b/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:40.611890 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aeac09-3c78-40d3-add0-9f98cc76ae68 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:40.612185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aeac09-3c78-40d3-add0-9f98cc76ae68 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:40.612469 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aeac09-3c78-40d3-add0-9f98cc76ae68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:40.633757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-f8aeac09-3c78-40d3-add0-9f98cc76ae68 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:40.635202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f8aeac09-3c78-40d3-add0-9f98cc76ae68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:40.714164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1047/2079] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:40.722887 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778e3da-90e2-47d9-836d-99e9329a824e admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:40.725017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-6778e3da-90e2-47d9-836d-99e9329a824e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:40.789336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778e3da-90e2-47d9-836d-99e9329a824e admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:40.794526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9c5fb-61f3-4f37-9f8a-84397d04ecce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:40.795413 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9c5fb-61f3-4f37-9f8a-84397d04ecce service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:40.795606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9c5fb-61f3-4f37-9f8a-84397d04ecce service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:40.795828 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9c5fb-61f3-4f37-9f8a-84397d04ecce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:40.805708 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9c5fb-61f3-4f37-9f8a-84397d04ecce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:40.807323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9c5fb-61f3-4f37-9f8a-84397d04ecce service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:40.822910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1033/2080] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:41.372694 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aeac09-3c78-40d3-add0-9f98cc76ae68 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:41.375572 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f8aeac09-3c78-40d3-add0-9f98cc76ae68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:41.388968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aeac09-3c78-40d3-add0-9f98cc76ae68 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:41.395746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb73e4f-6e1e-4620-bbde-52c0bdd58c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:41.397023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb73e4f-6e1e-4620-bbde-52c0bdd58c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:41.397566 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb73e4f-6e1e-4620-bbde-52c0bdd58c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:41.397989 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb73e4f-6e1e-4620-bbde-52c0bdd58c0c admin admin] PATH_INFO: `/v3/users/4ba98e87b9d4481aa222a61dc3c29cad` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:41.399884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb73e4f-6e1e-4620-bbde-52c0bdd58c0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ba98e87b9d4481aa222a61dc3c29cad)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:41.401952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb73e4f-6e1e-4620-bbde-52c0bdd58c0c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:41.458874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-feb73e4f-6e1e-4620-bbde-52c0bdd58c0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ba98e87b9d4481aa222a61dc3c29cad'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:41.472093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-feb73e4f-6e1e-4620-bbde-52c0bdd58c0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ba98e87b9d4481aa222a61dc3c29cad'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:41.473879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-feb73e4f-6e1e-4620-bbde-52c0bdd58c0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ba98e87b9d4481aa222a61dc3c29cad'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:41.475351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-feb73e4f-6e1e-4620-bbde-52c0bdd58c0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ba98e87b9d4481aa222a61dc3c29cad'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:41.484057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-feb73e4f-6e1e-4620-bbde-52c0bdd58c0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ba98e87b9d4481aa222a61dc3c29cad'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:41.500190 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1048/2081] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:41 2022] DELETE /identity/v3/users/4ba98e87b9d4481aa222a61dc3c29cad => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:41.504828 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536633-f7ff-4ffc-a837-109da4478cc3 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:41.505306 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536633-f7ff-4ffc-a837-109da4478cc3 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:41.505396 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536633-f7ff-4ffc-a837-109da4478cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:41.524742 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-2a536633-f7ff-4ffc-a837-109da4478cc3 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:41.525943 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2a536633-f7ff-4ffc-a837-109da4478cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:41.626549 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1034/2082] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:41.643890 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb73e4f-6e1e-4620-bbde-52c0bdd58c0c admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:41.645673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-feb73e4f-6e1e-4620-bbde-52c0bdd58c0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:41.725541 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb73e4f-6e1e-4620-bbde-52c0bdd58c0c admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:41.729609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ffa50-261b-4c75-a672-c91917c154c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:41.730261 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ffa50-261b-4c75-a672-c91917c154c2 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:41.730548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ffa50-261b-4c75-a672-c91917c154c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:41.730964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ffa50-261b-4c75-a672-c91917c154c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:41.743204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ffa50-261b-4c75-a672-c91917c154c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:41.744896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ffa50-261b-4c75-a672-c91917c154c2 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:41.764620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1049/2083] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:41.909121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fc662-c503-4d53-97fa-1a1d3c24843a None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:41.909566 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fc662-c503-4d53-97fa-1a1d3c24843a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:41.910571 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fc662-c503-4d53-97fa-1a1d3c24843a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:42.012391 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-534fc662-c503-4d53-97fa-1a1d3c24843a None None] MFA Rules not processed for user `004034ab6aca41ae893214bf0e81841b`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:42.013662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-534fc662-c503-4d53-97fa-1a1d3c24843a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:42.139652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1035/2084] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:42.156801 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ffa50-261b-4c75-a672-c91917c154c2 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:42.158058 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-045ffa50-261b-4c75-a672-c91917c154c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:42.224607 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ffa50-261b-4c75-a672-c91917c154c2 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:42.229057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e3610-db4d-4744-ba74-d479cb164292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:42.229963 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e3610-db4d-4744-ba74-d479cb164292 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:42.230316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e3610-db4d-4744-ba74-d479cb164292 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:42.230665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e3610-db4d-4744-ba74-d479cb164292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:42.240941 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e3610-db4d-4744-ba74-d479cb164292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:42.242482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e3610-db4d-4744-ba74-d479cb164292 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:42.263079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1050/2085] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:42.550848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fc662-c503-4d53-97fa-1a1d3c24843a None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:42.553849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-534fc662-c503-4d53-97fa-1a1d3c24843a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:42.579943 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fc662-c503-4d53-97fa-1a1d3c24843a None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:42.592009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:42.593758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:42.594264 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:42.594915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] PATH_INFO: `/v3/projects/e5c046789e39424cbba3c7e523af0c43` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:42.597325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5c046789e39424cbba3c7e523af0c43)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:42.600281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:42.724750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5c046789e39424cbba3c7e523af0c43', 'request_id': 'req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:42.732837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5c046789e39424cbba3c7e523af0c43', 'request_id': 'req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:42.744134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5c046789e39424cbba3c7e523af0c43', 'request_id': 'req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:42.764294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] The token cache is being invalidate because project e5c046789e39424cbba3c7e523af0c43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:42.764584 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:42.765746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:42.767715 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1036/2086] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:42 2022] DELETE /identity/v3/projects/e5c046789e39424cbba3c7e523af0c43 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:42.772846 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e3610-db4d-4744-ba74-d479cb164292 service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:42.785807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e3610-db4d-4744-ba74-d479cb164292 service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:42.863585 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e5d8d-4808-40cd-875f-e83e3e91ccae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:42.864472 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e5d8d-4808-40cd-875f-e83e3e91ccae admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:42.864753 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e5d8d-4808-40cd-875f-e83e3e91ccae admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:42.865026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e5d8d-4808-40cd-875f-e83e3e91ccae admin admin] PATH_INFO: `/v3/users/fe16fb4c480a405eb4b429fcb8b8e796` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:42.866786 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e5d8d-4808-40cd-875f-e83e3e91ccae admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe16fb4c480a405eb4b429fcb8b8e796)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:42.869504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e5d8d-4808-40cd-875f-e83e3e91ccae admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:42.940338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a12e5d8d-4808-40cd-875f-e83e3e91ccae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe16fb4c480a405eb4b429fcb8b8e796'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:42.959820 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a12e5d8d-4808-40cd-875f-e83e3e91ccae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe16fb4c480a405eb4b429fcb8b8e796'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:42.961153 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a12e5d8d-4808-40cd-875f-e83e3e91ccae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe16fb4c480a405eb4b429fcb8b8e796'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:42.961789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a12e5d8d-4808-40cd-875f-e83e3e91ccae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe16fb4c480a405eb4b429fcb8b8e796'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:42.969076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a12e5d8d-4808-40cd-875f-e83e3e91ccae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe16fb4c480a405eb4b429fcb8b8e796'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:42.992366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1051/2087] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:42 2022] DELETE /identity/v3/users/fe16fb4c480a405eb4b429fcb8b8e796 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:43.505068 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:43.506585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:43.521125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26815938-3d08-4d99-ab1d-b3ff1ec75d6b admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:43.582929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368ad8b-bd5b-40f7-bd36-3abf99e61577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:43.583872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368ad8b-bd5b-40f7-bd36-3abf99e61577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:43.584163 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368ad8b-bd5b-40f7-bd36-3abf99e61577 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:43.584442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368ad8b-bd5b-40f7-bd36-3abf99e61577 admin admin] PATH_INFO: `/v3/projects/4850349064874e689dbff9e0ff497321` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:43.585691 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368ad8b-bd5b-40f7-bd36-3abf99e61577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4850349064874e689dbff9e0ff497321)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:43.587632 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368ad8b-bd5b-40f7-bd36-3abf99e61577 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:43.649125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5368ad8b-bd5b-40f7-bd36-3abf99e61577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4850349064874e689dbff9e0ff497321', 'request_id': 'req-5368ad8b-bd5b-40f7-bd36-3abf99e61577'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:43.659237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5368ad8b-bd5b-40f7-bd36-3abf99e61577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4850349064874e689dbff9e0ff497321', 'request_id': 'req-5368ad8b-bd5b-40f7-bd36-3abf99e61577'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:43.675922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5368ad8b-bd5b-40f7-bd36-3abf99e61577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4850349064874e689dbff9e0ff497321', 'request_id': 'req-5368ad8b-bd5b-40f7-bd36-3abf99e61577'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:43.690670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5368ad8b-bd5b-40f7-bd36-3abf99e61577 admin admin] The token cache is being invalidate because project 4850349064874e689dbff9e0ff497321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:43.691774 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5368ad8b-bd5b-40f7-bd36-3abf99e61577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:43.692764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5368ad8b-bd5b-40f7-bd36-3abf99e61577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:43.696769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1037/2088] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:43 2022] DELETE /identity/v3/projects/4850349064874e689dbff9e0ff497321 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:43.706441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd215c-7c7a-4373-b9d5-1432c390a4a6 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:43.706935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd215c-7c7a-4373-b9d5-1432c390a4a6 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:43.707221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd215c-7c7a-4373-b9d5-1432c390a4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:43.751426 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-74cd215c-7c7a-4373-b9d5-1432c390a4a6 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:43.752609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-74cd215c-7c7a-4373-b9d5-1432c390a4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:43.835931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1052/2089] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:43.837834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22802544-59f9-4395-b058-722ba2b64a16 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:43.838129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22802544-59f9-4395-b058-722ba2b64a16 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:43.838488 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22802544-59f9-4395-b058-722ba2b64a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:43.868866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-22802544-59f9-4395-b058-722ba2b64a16 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:43.870779 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-22802544-59f9-4395-b058-722ba2b64a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:43.881198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1038/2090] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:43.885518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd215c-7c7a-4373-b9d5-1432c390a4a6 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:43.896415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd215c-7c7a-4373-b9d5-1432c390a4a6 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:43.902577 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a9700-cfdb-4b58-8b3c-3e2c45e24bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:43.903544 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a9700-cfdb-4b58-8b3c-3e2c45e24bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:43.903835 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a9700-cfdb-4b58-8b3c-3e2c45e24bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:43.904681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a9700-cfdb-4b58-8b3c-3e2c45e24bb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:43.905464 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a9700-cfdb-4b58-8b3c-3e2c45e24bb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:43.905828 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a9700-cfdb-4b58-8b3c-3e2c45e24bb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:43.907897 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a9700-cfdb-4b58-8b3c-3e2c45e24bb0 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:43.928380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1053/2091] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:43.930426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22802544-59f9-4395-b058-722ba2b64a16 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:43.943787 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22802544-59f9-4395-b058-722ba2b64a16 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:43.960445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a8f4f-eed3-4920-a1f6-2b3e15867b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:43.961709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a8f4f-eed3-4920-a1f6-2b3e15867b52 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:43.962327 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a8f4f-eed3-4920-a1f6-2b3e15867b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:43.962631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a8f4f-eed3-4920-a1f6-2b3e15867b52 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:43.963904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a8f4f-eed3-4920-a1f6-2b3e15867b52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:43.967044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a8f4f-eed3-4920-a1f6-2b3e15867b52 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:44.017917 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1039/2092] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:43 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:44.019097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a9700-cfdb-4b58-8b3c-3e2c45e24bb0 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:44.031595 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a9700-cfdb-4b58-8b3c-3e2c45e24bb0 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:44.099028 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162c7cb-9dce-41f9-920c-8adbda642aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:44.111364 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162c7cb-9dce-41f9-920c-8adbda642aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:44.112165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162c7cb-9dce-41f9-920c-8adbda642aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:44.112605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162c7cb-9dce-41f9-920c-8adbda642aa1 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:44.115030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162c7cb-9dce-41f9-920c-8adbda642aa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:44.121888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162c7cb-9dce-41f9-920c-8adbda642aa1 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:44.165416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1054/2093] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:44 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:44.174598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a8f4f-eed3-4920-a1f6-2b3e15867b52 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:44.203672 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a8f4f-eed3-4920-a1f6-2b3e15867b52 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:44.209648 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55ba00-465c-419c-97f3-b7aa0fea11f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:44.210969 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ba00-465c-419c-97f3-b7aa0fea11f8 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:44.211201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ba00-465c-419c-97f3-b7aa0fea11f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:44.211438 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ba00-465c-419c-97f3-b7aa0fea11f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:44.212176 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55ba00-465c-419c-97f3-b7aa0fea11f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:44.215518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55ba00-465c-419c-97f3-b7aa0fea11f8 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:44.235700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1040/2094] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:44.239047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162c7cb-9dce-41f9-920c-8adbda642aa1 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:44.254976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162c7cb-9dce-41f9-920c-8adbda642aa1 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:44.261661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c16707-997c-4436-9897-c603b496b847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:44.262617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c16707-997c-4436-9897-c603b496b847 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:44.263002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c16707-997c-4436-9897-c603b496b847 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:44.263457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c16707-997c-4436-9897-c603b496b847 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:44.264378 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c16707-997c-4436-9897-c603b496b847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:44.266509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c16707-997c-4436-9897-c603b496b847 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:44.311808 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1055/2095] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:44.313673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55ba00-465c-419c-97f3-b7aa0fea11f8 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:44.329735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55ba00-465c-419c-97f3-b7aa0fea11f8 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:44.347083 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86a404-bf24-4747-8f2b-6a9e4619c5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:44.348556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86a404-bf24-4747-8f2b-6a9e4619c5c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:44.348843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86a404-bf24-4747-8f2b-6a9e4619c5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:44.349185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86a404-bf24-4747-8f2b-6a9e4619c5c1 admin admin] PATH_INFO: `/v3/projects/a072427737e24114ab02836dfaae572f/users/0d9a0a87922c4b5d917ed7ac2edfa6eb/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:44.373363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86a404-bf24-4747-8f2b-6a9e4619c5c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a072427737e24114ab02836dfaae572f, user_id=0d9a0a87922c4b5d917ed7ac2edfa6eb, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:44.376217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86a404-bf24-4747-8f2b-6a9e4619c5c1 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:44.408785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1041/2096] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:44 2022] PUT /identity/v3/projects/a072427737e24114ab02836dfaae572f/users/0d9a0a87922c4b5d917ed7ac2edfa6eb/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:44.410629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd074b-532c-418c-b897-2f64df537249 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:44.410629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd074b-532c-418c-b897-2f64df537249 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:44.410629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd074b-532c-418c-b897-2f64df537249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:44.463726 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d8dd074b-532c-418c-b897-2f64df537249 None None] MFA Rules not processed for user `0d9a0a87922c4b5d917ed7ac2edfa6eb`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:44.465661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d8dd074b-532c-418c-b897-2f64df537249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:44.668552 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1056/2097] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:44.678994 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86a404-bf24-4747-8f2b-6a9e4619c5c1 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:44.680923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-3b86a404-bf24-4747-8f2b-6a9e4619c5c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:44.794537 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86a404-bf24-4747-8f2b-6a9e4619c5c1 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:44.805990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d5d3b-8081-41f4-93a8-e32ffc4b355c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:44.807021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d5d3b-8081-41f4-93a8-e32ffc4b355c service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:44.807387 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d5d3b-8081-41f4-93a8-e32ffc4b355c service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:44.808094 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d5d3b-8081-41f4-93a8-e32ffc4b355c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:44.829989 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d5d3b-8081-41f4-93a8-e32ffc4b355c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:44.831549 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d5d3b-8081-41f4-93a8-e32ffc4b355c service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:44.850462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1042/2098] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:44.875130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd074b-532c-418c-b897-2f64df537249 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:44.909477 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2c2b7-ef4d-4266-923a-064f9437eced None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:44.909763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2c2b7-ef4d-4266-923a-064f9437eced None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:44.910146 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2c2b7-ef4d-4266-923a-064f9437eced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:44.956857 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-f6c2c2b7-ef4d-4266-923a-064f9437eced None None] MFA Rules not processed for user `004034ab6aca41ae893214bf0e81841b`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:44.958087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f6c2c2b7-ef4d-4266-923a-064f9437eced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:45.049943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd074b-532c-418c-b897-2f64df537249 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:45.060218 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d3d84-a13a-44be-bf5b-c538c818633d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:45.063213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d3d84-a13a-44be-bf5b-c538c818633d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:45.070514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d3d84-a13a-44be-bf5b-c538c818633d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:45.071432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d3d84-a13a-44be-bf5b-c538c818633d admin admin] PATH_INFO: `/v3/users/0d9a0a87922c4b5d917ed7ac2edfa6eb` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:45.081894 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d3d84-a13a-44be-bf5b-c538c818633d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d9a0a87922c4b5d917ed7ac2edfa6eb)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:45.084730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d3d84-a13a-44be-bf5b-c538c818633d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:45.110244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1043/2099] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:45.229154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3d7d3d84-a13a-44be-bf5b-c538c818633d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d9a0a87922c4b5d917ed7ac2edfa6eb'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:45.254902 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3d7d3d84-a13a-44be-bf5b-c538c818633d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d9a0a87922c4b5d917ed7ac2edfa6eb'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:45.255811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3d7d3d84-a13a-44be-bf5b-c538c818633d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d9a0a87922c4b5d917ed7ac2edfa6eb'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:45.256650 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3d7d3d84-a13a-44be-bf5b-c538c818633d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d9a0a87922c4b5d917ed7ac2edfa6eb'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:45.264932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3d7d3d84-a13a-44be-bf5b-c538c818633d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d9a0a87922c4b5d917ed7ac2edfa6eb'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:45.292955 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1057/2100] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:44 2022] DELETE /identity/v3/users/0d9a0a87922c4b5d917ed7ac2edfa6eb => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:45.298019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cca560-def6-47b2-b271-0f02ca146973 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:45.298275 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cca560-def6-47b2-b271-0f02ca146973 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:45.298550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cca560-def6-47b2-b271-0f02ca146973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:45.333617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-80cca560-def6-47b2-b271-0f02ca146973 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:45.334912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-80cca560-def6-47b2-b271-0f02ca146973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:45.467987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1044/2101] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:45.499365 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d3d84-a13a-44be-bf5b-c538c818633d admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:45.503979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3d7d3d84-a13a-44be-bf5b-c538c818633d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:45.547355 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cca560-def6-47b2-b271-0f02ca146973 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:45.549162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-80cca560-def6-47b2-b271-0f02ca146973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:45.573675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cca560-def6-47b2-b271-0f02ca146973 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:45.585231 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8aea26-1add-409e-a61c-baef05373365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:45.586301 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8aea26-1add-409e-a61c-baef05373365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:45.586868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8aea26-1add-409e-a61c-baef05373365 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:45.587322 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8aea26-1add-409e-a61c-baef05373365 admin admin] PATH_INFO: `/v3/users/1315f98467d8426094229e3a31c618f3` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:45.588585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8aea26-1add-409e-a61c-baef05373365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1315f98467d8426094229e3a31c618f3)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:45.590533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8aea26-1add-409e-a61c-baef05373365 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:45.605823 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d3d84-a13a-44be-bf5b-c538c818633d admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:45.610900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150bdc2-4193-4308-be46-7373d725029a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:45.611685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150bdc2-4193-4308-be46-7373d725029a service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:45.612033 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150bdc2-4193-4308-be46-7373d725029a service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:45.612455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150bdc2-4193-4308-be46-7373d725029a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:45.625669 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150bdc2-4193-4308-be46-7373d725029a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:45.632314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150bdc2-4193-4308-be46-7373d725029a service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:45.657261 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1058/2102] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:45.674566 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-cb8aea26-1add-409e-a61c-baef05373365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1315f98467d8426094229e3a31c618f3'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:45.691149 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-cb8aea26-1add-409e-a61c-baef05373365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1315f98467d8426094229e3a31c618f3'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:45.692257 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-cb8aea26-1add-409e-a61c-baef05373365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1315f98467d8426094229e3a31c618f3'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:45.693331 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-cb8aea26-1add-409e-a61c-baef05373365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1315f98467d8426094229e3a31c618f3'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:45.703413 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-cb8aea26-1add-409e-a61c-baef05373365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1315f98467d8426094229e3a31c618f3'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:45.757435 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1045/2103] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:45 2022] DELETE /identity/v3/users/1315f98467d8426094229e3a31c618f3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:46.114939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150bdc2-4193-4308-be46-7373d725029a service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:46.130115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150bdc2-4193-4308-be46-7373d725029a service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:46.209556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20225b56-9d02-401e-90f6-c8591db0d47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:46.210804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20225b56-9d02-401e-90f6-c8591db0d47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:46.211246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20225b56-9d02-401e-90f6-c8591db0d47e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:46.211525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20225b56-9d02-401e-90f6-c8591db0d47e admin admin] PATH_INFO: `/v3/projects/39174f764455497c99b2fc47838407f8` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:46.212772 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20225b56-9d02-401e-90f6-c8591db0d47e admin admin] RBAC: Authorizing `identity:delete_project(project_id=39174f764455497c99b2fc47838407f8)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:46.214983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20225b56-9d02-401e-90f6-c8591db0d47e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:46.282696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-20225b56-9d02-401e-90f6-c8591db0d47e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39174f764455497c99b2fc47838407f8', 'request_id': 'req-20225b56-9d02-401e-90f6-c8591db0d47e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:46.291766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-20225b56-9d02-401e-90f6-c8591db0d47e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39174f764455497c99b2fc47838407f8', 'request_id': 'req-20225b56-9d02-401e-90f6-c8591db0d47e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:46.309534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-20225b56-9d02-401e-90f6-c8591db0d47e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39174f764455497c99b2fc47838407f8', 'request_id': 'req-20225b56-9d02-401e-90f6-c8591db0d47e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:46.323963 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-20225b56-9d02-401e-90f6-c8591db0d47e admin admin] The token cache is being invalidate because project 39174f764455497c99b2fc47838407f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:46.324283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-20225b56-9d02-401e-90f6-c8591db0d47e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:46.326414 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-20225b56-9d02-401e-90f6-c8591db0d47e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:46.331969 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1059/2104] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:46 2022] DELETE /identity/v3/projects/39174f764455497c99b2fc47838407f8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:46.334055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b42d40-7f47-44c7-befc-786b80662efc None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:46.334283 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b42d40-7f47-44c7-befc-786b80662efc None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:46.334589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b42d40-7f47-44c7-befc-786b80662efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:46.359704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-90b42d40-7f47-44c7-befc-786b80662efc None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:46.360939 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-90b42d40-7f47-44c7-befc-786b80662efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:46.453466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1046/2105] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:46.465311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13ac5b-bb7d-4b79-92bf-4337708313ba None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:46.465924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13ac5b-bb7d-4b79-92bf-4337708313ba None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:46.466482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13ac5b-bb7d-4b79-92bf-4337708313ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:46.479158 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b42d40-7f47-44c7-befc-786b80662efc None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:46.483811 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-90b42d40-7f47-44c7-befc-786b80662efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:46.498204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-ba13ac5b-bb7d-4b79-92bf-4337708313ba None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:46.523885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ba13ac5b-bb7d-4b79-92bf-4337708313ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:46.524679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b42d40-7f47-44c7-befc-786b80662efc None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:46.537447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be500302-6736-4252-8000-2c7d5c47bcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:46.538608 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be500302-6736-4252-8000-2c7d5c47bcbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:46.538927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be500302-6736-4252-8000-2c7d5c47bcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:46.539363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be500302-6736-4252-8000-2c7d5c47bcbb admin admin] PATH_INFO: `/v3/projects/a072427737e24114ab02836dfaae572f` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:46.543406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be500302-6736-4252-8000-2c7d5c47bcbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a072427737e24114ab02836dfaae572f)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:46.546457 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be500302-6736-4252-8000-2c7d5c47bcbb admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:46.566951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1060/2106] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:46.572119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13ac5b-bb7d-4b79-92bf-4337708313ba None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:46.601213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13ac5b-bb7d-4b79-92bf-4337708313ba None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:46.615827 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c83a3-b69f-45c4-9ba7-8b596c737d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:46.618370 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c83a3-b69f-45c4-9ba7-8b596c737d30 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:46.619624 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c83a3-b69f-45c4-9ba7-8b596c737d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:46.619983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c83a3-b69f-45c4-9ba7-8b596c737d30 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:46.627062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c83a3-b69f-45c4-9ba7-8b596c737d30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:46.630023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c83a3-b69f-45c4-9ba7-8b596c737d30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:46.632630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c83a3-b69f-45c4-9ba7-8b596c737d30 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:46.648718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1061/2107] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:46.653184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c83a3-b69f-45c4-9ba7-8b596c737d30 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:46.675280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-be500302-6736-4252-8000-2c7d5c47bcbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a072427737e24114ab02836dfaae572f', 'request_id': 'req-be500302-6736-4252-8000-2c7d5c47bcbb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:46.679480 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c83a3-b69f-45c4-9ba7-8b596c737d30 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:46.695792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-be500302-6736-4252-8000-2c7d5c47bcbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a072427737e24114ab02836dfaae572f', 'request_id': 'req-be500302-6736-4252-8000-2c7d5c47bcbb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:46.717770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-be500302-6736-4252-8000-2c7d5c47bcbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a072427737e24114ab02836dfaae572f', 'request_id': 'req-be500302-6736-4252-8000-2c7d5c47bcbb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:46.730576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-be500302-6736-4252-8000-2c7d5c47bcbb admin admin] The token cache is being invalidate because project a072427737e24114ab02836dfaae572f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:46.737576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-be500302-6736-4252-8000-2c7d5c47bcbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:46.738380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-be500302-6736-4252-8000-2c7d5c47bcbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:46.740090 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1047/2108] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:46 2022] DELETE /identity/v3/projects/a072427737e24114ab02836dfaae572f => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:46.745196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c7312-06b6-4c40-a4a4-613c53b203f6 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:46.745559 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c7312-06b6-4c40-a4a4-613c53b203f6 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:46.746604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c7312-06b6-4c40-a4a4-613c53b203f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:46.790759 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707fb608-80b2-4e0d-9114-c3ce0a209585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:46.791429 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fb608-80b2-4e0d-9114-c3ce0a209585 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:46.791710 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fb608-80b2-4e0d-9114-c3ce0a209585 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:46.791985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fb608-80b2-4e0d-9114-c3ce0a209585 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:46.792689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707fb608-80b2-4e0d-9114-c3ce0a209585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:46.796186 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-587c7312-06b6-4c40-a4a4-613c53b203f6 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:46.798022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-587c7312-06b6-4c40-a4a4-613c53b203f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:46.800294 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707fb608-80b2-4e0d-9114-c3ce0a209585 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:46.820746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1048/2109] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:46.823821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ba09-4680-48e6-ad50-57db5f6598f6 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:46.824115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ba09-4680-48e6-ad50-57db5f6598f6 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:46.824444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ba09-4680-48e6-ad50-57db5f6598f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:46.856671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1062/2110] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:46 2022] POST /identity/v3/projects => generated 375 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:46.862172 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-68d5ba09-4680-48e6-ad50-57db5f6598f6 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:46.862449 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707fb608-80b2-4e0d-9114-c3ce0a209585 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:46.863841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-68d5ba09-4680-48e6-ad50-57db5f6598f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:46.864789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-707fb608-80b2-4e0d-9114-c3ce0a209585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:46.993864 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707fb608-80b2-4e0d-9114-c3ce0a209585 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:47.008622 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef728d18-2986-4ccd-91cd-e52925b8c7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:47.009557 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef728d18-2986-4ccd-91cd-e52925b8c7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:47.009947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef728d18-2986-4ccd-91cd-e52925b8c7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:47.010209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef728d18-2986-4ccd-91cd-e52925b8c7e4 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:47.010311 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1049/2111] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:47.011494 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef728d18-2986-4ccd-91cd-e52925b8c7e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:47.012308 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5ba09-4680-48e6-ad50-57db5f6598f6 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:47.017330 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef728d18-2986-4ccd-91cd-e52925b8c7e4 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:47.032106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5ba09-4680-48e6-ad50-57db5f6598f6 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:47.050364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07986e-3a5a-4c45-8314-f8b1b3ca93f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:47.051320 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07986e-3a5a-4c45-8314-f8b1b3ca93f3 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:47.051862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07986e-3a5a-4c45-8314-f8b1b3ca93f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:47.052254 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07986e-3a5a-4c45-8314-f8b1b3ca93f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:47.053195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07986e-3a5a-4c45-8314-f8b1b3ca93f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:47.053617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07986e-3a5a-4c45-8314-f8b1b3ca93f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:47.054360 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1063/2112] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:46 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:47.055579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07986e-3a5a-4c45-8314-f8b1b3ca93f3 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:47.058893 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef728d18-2986-4ccd-91cd-e52925b8c7e4 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:47.071166 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef728d18-2986-4ccd-91cd-e52925b8c7e4 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:47.072090 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1050/2113] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:47.076949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07986e-3a5a-4c45-8314-f8b1b3ca93f3 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:47.077990 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc312e73-5651-4171-80c8-581154a57687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:47.078770 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc312e73-5651-4171-80c8-581154a57687 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:47.079055 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc312e73-5651-4171-80c8-581154a57687 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:47.079386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc312e73-5651-4171-80c8-581154a57687 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:47.080040 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc312e73-5651-4171-80c8-581154a57687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:47.081666 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc312e73-5651-4171-80c8-581154a57687 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:47.089987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07986e-3a5a-4c45-8314-f8b1b3ca93f3 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:47.096296 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7935e-4415-4c74-a52a-617d0c4d99d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:47.097280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7935e-4415-4c74-a52a-617d0c4d99d9 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:47.097407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7935e-4415-4c74-a52a-617d0c4d99d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:47.097664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7935e-4415-4c74-a52a-617d0c4d99d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:47.098429 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7935e-4415-4c74-a52a-617d0c4d99d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:47.100730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7935e-4415-4c74-a52a-617d0c4d99d9 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:47.103443 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1064/2114] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:47.109712 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc312e73-5651-4171-80c8-581154a57687 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:47.123108 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc312e73-5651-4171-80c8-581154a57687 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:47.130520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a4beb-cc9b-4ade-9efd-5ed88b4bc51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:47.131513 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a4beb-cc9b-4ade-9efd-5ed88b4bc51b admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:47.132057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a4beb-cc9b-4ade-9efd-5ed88b4bc51b admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:47.132631 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a4beb-cc9b-4ade-9efd-5ed88b4bc51b admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:47.133814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a4beb-cc9b-4ade-9efd-5ed88b4bc51b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:47.135996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a4beb-cc9b-4ade-9efd-5ed88b4bc51b admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:47.161316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1065/2115] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:52:47.166666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1051/2116] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:47 2022] POST /identity/v3/projects => generated 377 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:47.167063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a4beb-cc9b-4ade-9efd-5ed88b4bc51b admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:47.168371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7935e-4415-4c74-a52a-617d0c4d99d9 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:47.199144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a4beb-cc9b-4ade-9efd-5ed88b4bc51b admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:47.354876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7935e-4415-4c74-a52a-617d0c4d99d9 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:47.359396 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b66d7-77f2-4717-887d-10b439bd483b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:47.362337 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b66d7-77f2-4717-887d-10b439bd483b admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:47.364238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b66d7-77f2-4717-887d-10b439bd483b admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:47.364894 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b66d7-77f2-4717-887d-10b439bd483b admin admin] PATH_INFO: `/v3/projects/c0e44b87f674490a9d95d2ae148a0e1a/users/f82a1348def44e7d8978431779cf360c/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:47.375400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7894dd4-1bcc-4413-8244-7d6319d6f414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:47.377637 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7894dd4-1bcc-4413-8244-7d6319d6f414 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:47.378014 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7894dd4-1bcc-4413-8244-7d6319d6f414 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:47.378298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7894dd4-1bcc-4413-8244-7d6319d6f414 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:47.379491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7894dd4-1bcc-4413-8244-7d6319d6f414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:47.384284 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7894dd4-1bcc-4413-8244-7d6319d6f414 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:47.405541 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b66d7-77f2-4717-887d-10b439bd483b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0e44b87f674490a9d95d2ae148a0e1a, user_id=f82a1348def44e7d8978431779cf360c, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:47.413748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b66d7-77f2-4717-887d-10b439bd483b admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:47.434250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1052/2117] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:47 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:47.436323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7894dd4-1bcc-4413-8244-7d6319d6f414 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:47.440918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1066/2118] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:47 2022] PUT /identity/v3/projects/c0e44b87f674490a9d95d2ae148a0e1a/users/f82a1348def44e7d8978431779cf360c/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:47.452861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccd27f-b683-468f-a241-c61698a33a7c None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:47.453315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccd27f-b683-468f-a241-c61698a33a7c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:47.453860 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccd27f-b683-468f-a241-c61698a33a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:47.464663 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7894dd4-1bcc-4413-8244-7d6319d6f414 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:47.511253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-30ccd27f-b683-468f-a241-c61698a33a7c None None] MFA Rules not processed for user `f82a1348def44e7d8978431779cf360c`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:47.513715 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-30ccd27f-b683-468f-a241-c61698a33a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:47.616670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae583cac-9964-40e4-b128-b6beaa24a9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:47.617367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae583cac-9964-40e4-b128-b6beaa24a9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:47.617648 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae583cac-9964-40e4-b128-b6beaa24a9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:47.617926 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae583cac-9964-40e4-b128-b6beaa24a9a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:47.618663 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae583cac-9964-40e4-b128-b6beaa24a9a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:47.620308 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae583cac-9964-40e4-b128-b6beaa24a9a7 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:47.630132 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1067/2119] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:47.650763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1053/2120] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:47.652190 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccd27f-b683-468f-a241-c61698a33a7c None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:47.663634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae583cac-9964-40e4-b128-b6beaa24a9a7 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:47.667617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccd27f-b683-468f-a241-c61698a33a7c None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:47.668755 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ae583cac-9964-40e4-b128-b6beaa24a9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:47.674782 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e90321-1924-4e77-a9fe-a1201f9e1a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:47.675522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e90321-1924-4e77-a9fe-a1201f9e1a15 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:47.675809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e90321-1924-4e77-a9fe-a1201f9e1a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:47.676091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e90321-1924-4e77-a9fe-a1201f9e1a15 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:47.676825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e90321-1924-4e77-a9fe-a1201f9e1a15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:47.678643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e90321-1924-4e77-a9fe-a1201f9e1a15 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:47.696551 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1068/2121] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:47.697525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e90321-1924-4e77-a9fe-a1201f9e1a15 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:47.709535 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e90321-1924-4e77-a9fe-a1201f9e1a15 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:47.715408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565b2c1-ad52-4577-a233-51b062cffed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:47.716392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565b2c1-ad52-4577-a233-51b062cffed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:47.716872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565b2c1-ad52-4577-a233-51b062cffed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:47.717225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565b2c1-ad52-4577-a233-51b062cffed3 admin admin] PATH_INFO: `/v3/projects/0e4da8602d7140f5a3feb7713b96aef8/users/8784e340a42f40fbba60fefa4ef50055/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:47.744131 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565b2c1-ad52-4577-a233-51b062cffed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e4da8602d7140f5a3feb7713b96aef8, user_id=8784e340a42f40fbba60fefa4ef50055, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:47.748352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565b2c1-ad52-4577-a233-51b062cffed3 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:47.779170 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1069/2122] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:47 2022] PUT /identity/v3/projects/0e4da8602d7140f5a3feb7713b96aef8/users/8784e340a42f40fbba60fefa4ef50055/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:47.789454 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9de44-a843-4992-aae8-462cb4fed448 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:47.790415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9de44-a843-4992-aae8-462cb4fed448 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:47.790912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9de44-a843-4992-aae8-462cb4fed448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:47.836349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae583cac-9964-40e4-b128-b6beaa24a9a7 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:47.838242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-5ca9de44-a843-4992-aae8-462cb4fed448 None None] MFA Rules not processed for user `8784e340a42f40fbba60fefa4ef50055`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:47.839499 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-5ca9de44-a843-4992-aae8-462cb4fed448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:47.912153 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea026f1-9cd2-46f8-8511-3b402e8e81ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:47.913223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea026f1-9cd2-46f8-8511-3b402e8e81ba service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:47.913629 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea026f1-9cd2-46f8-8511-3b402e8e81ba service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:47.914008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea026f1-9cd2-46f8-8511-3b402e8e81ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:47.955227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1070/2123] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:47.957762 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c456f2e-f6fd-4c69-a241-10f3c46fc7a3 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:47.958430 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c456f2e-f6fd-4c69-a241-10f3c46fc7a3 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:47.958917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c456f2e-f6fd-4c69-a241-10f3c46fc7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:47.986930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7c456f2e-f6fd-4c69-a241-10f3c46fc7a3 None None] MFA Rules not processed for user `c3a0de9ca12746be893b97d3cd11b826`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:47.988244 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7c456f2e-f6fd-4c69-a241-10f3c46fc7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:47.989610 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea026f1-9cd2-46f8-8511-3b402e8e81ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:47.993090 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea026f1-9cd2-46f8-8511-3b402e8e81ba service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:48.005939 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1054/2124] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:52:48.006798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea026f1-9cd2-46f8-8511-3b402e8e81ba service nova] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:48.092408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1071/2125] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:48.093551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea026f1-9cd2-46f8-8511-3b402e8e81ba service nova] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:48.096788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c456f2e-f6fd-4c69-a241-10f3c46fc7a3 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:48.098229 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7c456f2e-f6fd-4c69-a241-10f3c46fc7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:48.098499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ac95e-bb34-4c07-9a50-d1114375dff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:48.099444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ac95e-bb34-4c07-9a50-d1114375dff5 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:48.099733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ac95e-bb34-4c07-9a50-d1114375dff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:48.100117 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ac95e-bb34-4c07-9a50-d1114375dff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:48.100900 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ac95e-bb34-4c07-9a50-d1114375dff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:48.103666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ac95e-bb34-4c07-9a50-d1114375dff5 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:48.118823 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c456f2e-f6fd-4c69-a241-10f3c46fc7a3 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:48.124876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810f1cc-b135-40a8-a864-1b0702ab125e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:48.125590 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810f1cc-b135-40a8-a864-1b0702ab125e admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:48.125818 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810f1cc-b135-40a8-a864-1b0702ab125e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:48.126094 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810f1cc-b135-40a8-a864-1b0702ab125e admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:48.126834 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810f1cc-b135-40a8-a864-1b0702ab125e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:48.129197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810f1cc-b135-40a8-a864-1b0702ab125e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:48.168707 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1055/2126] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:48 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:48.171879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ac95e-bb34-4c07-9a50-d1114375dff5 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:48.217558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1072/2127] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:48 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:48.222900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810f1cc-b135-40a8-a864-1b0702ab125e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:48.343859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ac95e-bb34-4c07-9a50-d1114375dff5 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:48.363790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810f1cc-b135-40a8-a864-1b0702ab125e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:48.370479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11b8e-d405-4dfa-a797-3c355ebdc9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:48.377023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11b8e-d405-4dfa-a797-3c355ebdc9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:48.377770 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11b8e-d405-4dfa-a797-3c355ebdc9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:48.392644 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11b8e-d405-4dfa-a797-3c355ebdc9a6 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:48.393634 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c11b8e-d405-4dfa-a797-3c355ebdc9a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:48.401922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c11b8e-d405-4dfa-a797-3c355ebdc9a6 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:48.411213 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd0a31-c677-41d0-90fc-f7aa6d617874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:48.417680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd0a31-c677-41d0-90fc-f7aa6d617874 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:48.417888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd0a31-c677-41d0-90fc-f7aa6d617874 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:48.418173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd0a31-c677-41d0-90fc-f7aa6d617874 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:48.418867 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd0a31-c677-41d0-90fc-f7aa6d617874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:48.421396 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd0a31-c677-41d0-90fc-f7aa6d617874 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:48.446726 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1073/2128] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:48 2022] POST /identity/v3/users => generated 295 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:48.449311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11b8e-d405-4dfa-a797-3c355ebdc9a6 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:48.454919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1056/2129] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:48 2022] POST /identity/v3/users => generated 289 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:48.455832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd0a31-c677-41d0-90fc-f7aa6d617874 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:48.467254 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd0a31-c677-41d0-90fc-f7aa6d617874 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:48.473565 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0854d-a50b-400a-9c91-e24642e7bffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:48.474534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0854d-a50b-400a-9c91-e24642e7bffd admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:48.474888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0854d-a50b-400a-9c91-e24642e7bffd admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:48.475176 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0854d-a50b-400a-9c91-e24642e7bffd admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:48.475922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0854d-a50b-400a-9c91-e24642e7bffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:48.477744 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0854d-a50b-400a-9c91-e24642e7bffd admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:48.478226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11b8e-d405-4dfa-a797-3c355ebdc9a6 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:48.492363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d927f55-3f11-4107-86a3-1dfcaba11a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:48.493410 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d927f55-3f11-4107-86a3-1dfcaba11a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:48.496981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1057/2130] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:48.497118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d927f55-3f11-4107-86a3-1dfcaba11a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:48.497118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d927f55-3f11-4107-86a3-1dfcaba11a51 admin admin] PATH_INFO: `/v3/users/f82a1348def44e7d8978431779cf360c` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:48.497309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0854d-a50b-400a-9c91-e24642e7bffd admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:48.507251 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0854d-a50b-400a-9c91-e24642e7bffd admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:48.508711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d927f55-3f11-4107-86a3-1dfcaba11a51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f82a1348def44e7d8978431779cf360c)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:48.515960 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480e773-5e74-45f0-bfae-e14d546ae3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:48.517179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480e773-5e74-45f0-bfae-e14d546ae3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:48.517179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480e773-5e74-45f0-bfae-e14d546ae3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:48.517420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480e773-5e74-45f0-bfae-e14d546ae3b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:48.526862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480e773-5e74-45f0-bfae-e14d546ae3b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:48.529543 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480e773-5e74-45f0-bfae-e14d546ae3b9 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:48.537972 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d927f55-3f11-4107-86a3-1dfcaba11a51 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:48.543347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1058/2131] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:48.544145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480e773-5e74-45f0-bfae-e14d546ae3b9 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:48.556839 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480e773-5e74-45f0-bfae-e14d546ae3b9 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:48.562307 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c902a-188f-4469-b591-e1b6914eed97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:48.563417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c902a-188f-4469-b591-e1b6914eed97 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:48.563721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c902a-188f-4469-b591-e1b6914eed97 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:48.563988 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c902a-188f-4469-b591-e1b6914eed97 admin admin] PATH_INFO: `/v3/projects/1db44b16e87747aeab1ff242b1556794/users/42a643613f144989b42be5865e82e4f2/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:48.581002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c902a-188f-4469-b591-e1b6914eed97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1db44b16e87747aeab1ff242b1556794, user_id=42a643613f144989b42be5865e82e4f2, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:48.584350 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c902a-188f-4469-b591-e1b6914eed97 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:48.609455 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1059/2132] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:48 2022] PUT /identity/v3/projects/1db44b16e87747aeab1ff242b1556794/users/42a643613f144989b42be5865e82e4f2/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:48.609455 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c902a-188f-4469-b591-e1b6914eed97 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:48.618209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9d927f55-3f11-4107-86a3-1dfcaba11a51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f82a1348def44e7d8978431779cf360c'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:48.643123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9d927f55-3f11-4107-86a3-1dfcaba11a51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f82a1348def44e7d8978431779cf360c'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:48.645400 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9d927f55-3f11-4107-86a3-1dfcaba11a51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f82a1348def44e7d8978431779cf360c'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:48.646524 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9d927f55-3f11-4107-86a3-1dfcaba11a51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f82a1348def44e7d8978431779cf360c'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:48.659419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9d927f55-3f11-4107-86a3-1dfcaba11a51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f82a1348def44e7d8978431779cf360c'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:48.678546 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1074/2133] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:48 2022] DELETE /identity/v3/users/f82a1348def44e7d8978431779cf360c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:48.679611 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d927f55-3f11-4107-86a3-1dfcaba11a51 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:48.681305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9d927f55-3f11-4107-86a3-1dfcaba11a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:48.738076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c902a-188f-4469-b591-e1b6914eed97 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:48.767272 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d927f55-3f11-4107-86a3-1dfcaba11a51 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:48.773426 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef62f8-18d6-4a98-9d65-feabfc60dd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:48.775968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef62f8-18d6-4a98-9d65-feabfc60dd10 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:48.776280 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef62f8-18d6-4a98-9d65-feabfc60dd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:48.776563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef62f8-18d6-4a98-9d65-feabfc60dd10 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:48.777316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef62f8-18d6-4a98-9d65-feabfc60dd10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:48.779912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef62f8-18d6-4a98-9d65-feabfc60dd10 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:48.798125 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1075/2134] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:48.799213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef62f8-18d6-4a98-9d65-feabfc60dd10 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:48.802280 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-1eef62f8-18d6-4a98-9d65-feabfc60dd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:48.817208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef62f8-18d6-4a98-9d65-feabfc60dd10 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:48.823023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e2d42-2ca5-47a1-83be-731f41e997da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:48.823940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e2d42-2ca5-47a1-83be-731f41e997da admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:48.824227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e2d42-2ca5-47a1-83be-731f41e997da admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:48.824503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e2d42-2ca5-47a1-83be-731f41e997da admin admin] PATH_INFO: `/v3/users/3c2857e783314386960a9970f4854f75` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:48.826819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e2d42-2ca5-47a1-83be-731f41e997da admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c2857e783314386960a9970f4854f75)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:48.829156 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e2d42-2ca5-47a1-83be-731f41e997da admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:48.834787 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307025b0-d9a1-401a-bcf9-e5abe443b833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:48.835747 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307025b0-d9a1-401a-bcf9-e5abe443b833 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:48.836195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307025b0-d9a1-401a-bcf9-e5abe443b833 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:48.837270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307025b0-d9a1-401a-bcf9-e5abe443b833 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:48.838006 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307025b0-d9a1-401a-bcf9-e5abe443b833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:48.839823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307025b0-d9a1-401a-bcf9-e5abe443b833 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:48.863024 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1060/2135] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:48.866682 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8812903-53bc-4b26-865f-271249713dc0 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:48.867272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8812903-53bc-4b26-865f-271249713dc0 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:48.867754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8812903-53bc-4b26-865f-271249713dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:48.914561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-c8812903-53bc-4b26-865f-271249713dc0 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:48.916051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c8812903-53bc-4b26-865f-271249713dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:48.926162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1061/2136] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:48.927520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8812903-53bc-4b26-865f-271249713dc0 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:48.936551 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-351e2d42-2ca5-47a1-83be-731f41e997da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c2857e783314386960a9970f4854f75'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:48.940619 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8812903-53bc-4b26-865f-271249713dc0 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:48.945943 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d1241-d268-4d50-80e0-4775e2bed0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:48.947134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d1241-d268-4d50-80e0-4775e2bed0ca admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:48.947794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d1241-d268-4d50-80e0-4775e2bed0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:48.948275 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d1241-d268-4d50-80e0-4775e2bed0ca admin admin] PATH_INFO: `/v3/projects/1db44b16e87747aeab1ff242b1556794/users/42a643613f144989b42be5865e82e4f2/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:48.951370 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d1241-d268-4d50-80e0-4775e2bed0ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=1db44b16e87747aeab1ff242b1556794, user_id=42a643613f144989b42be5865e82e4f2, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:48.955270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d1241-d268-4d50-80e0-4775e2bed0ca admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:48.959529 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-351e2d42-2ca5-47a1-83be-731f41e997da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c2857e783314386960a9970f4854f75'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:48.960676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-351e2d42-2ca5-47a1-83be-731f41e997da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c2857e783314386960a9970f4854f75'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:48.961435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-351e2d42-2ca5-47a1-83be-731f41e997da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c2857e783314386960a9970f4854f75'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:48.974307 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-351e2d42-2ca5-47a1-83be-731f41e997da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c2857e783314386960a9970f4854f75'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:48.975483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1062/2137] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:48 2022] PUT /identity/v3/projects/1db44b16e87747aeab1ff242b1556794/users/42a643613f144989b42be5865e82e4f2/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:48.976466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d1241-d268-4d50-80e0-4775e2bed0ca admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:48.978372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-e62d1241-d268-4d50-80e0-4775e2bed0ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:49.010229 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1076/2138] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:48 2022] DELETE /identity/v3/users/3c2857e783314386960a9970f4854f75 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:49.012911 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e2d42-2ca5-47a1-83be-731f41e997da admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:49.019906 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-351e2d42-2ca5-47a1-83be-731f41e997da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:49.142862 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e2d42-2ca5-47a1-83be-731f41e997da admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:49.161374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe4370-a6db-43e5-9879-f748ff3ef836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:49.162356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe4370-a6db-43e5-9879-f748ff3ef836 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:49.162852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe4370-a6db-43e5-9879-f748ff3ef836 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:49.163746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe4370-a6db-43e5-9879-f748ff3ef836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:49.168777 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-8ebe4370-a6db-43e5-9879-f748ff3ef836 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:49.175715 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d1241-d268-4d50-80e0-4775e2bed0ca admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:49.315100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81ba5a-5a37-4d79-a43d-edd5f6793344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:49.316242 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81ba5a-5a37-4d79-a43d-edd5f6793344 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:49.318021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81ba5a-5a37-4d79-a43d-edd5f6793344 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:49.318466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81ba5a-5a37-4d79-a43d-edd5f6793344 admin admin] PATH_INFO: `/v3/projects/33951e60f0cc4a2bb3efc3fdb9d03193/users/1bde078c591e47a39e7c360718667af1/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:49.356017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b81ba5a-5a37-4d79-a43d-edd5f6793344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33951e60f0cc4a2bb3efc3fdb9d03193, user_id=1bde078c591e47a39e7c360718667af1, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:49.359726 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe4370-a6db-43e5-9879-f748ff3ef836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:49.364217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b81ba5a-5a37-4d79-a43d-edd5f6793344 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:49.364994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe4370-a6db-43e5-9879-f748ff3ef836 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:49.388017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1077/2139] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:49.389414 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675fa53-6100-40ff-b2e3-1cad81dbe505 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:49.389703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675fa53-6100-40ff-b2e3-1cad81dbe505 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:49.389984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675fa53-6100-40ff-b2e3-1cad81dbe505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:49.390901 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1063/2140] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:48 2022] PUT /identity/v3/projects/33951e60f0cc4a2bb3efc3fdb9d03193/users/1bde078c591e47a39e7c360718667af1/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:49.401789 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b86d88-4bf6-4106-9652-0f96c9c74064 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:49.402195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b86d88-4bf6-4106-9652-0f96c9c74064 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:49.402544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b86d88-4bf6-4106-9652-0f96c9c74064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:49.443518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e675fa53-6100-40ff-b2e3-1cad81dbe505 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:49.446550 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e675fa53-6100-40ff-b2e3-1cad81dbe505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:49.465611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-a2b86d88-4bf6-4106-9652-0f96c9c74064 None None] MFA Rules not processed for user `1bde078c591e47a39e7c360718667af1`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:49.467972 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a2b86d88-4bf6-4106-9652-0f96c9c74064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:49.594661 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1064/2141] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:49.595868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b86d88-4bf6-4106-9652-0f96c9c74064 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:49.663398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1078/2142] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:49.676744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675fa53-6100-40ff-b2e3-1cad81dbe505 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:49.679001 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e675fa53-6100-40ff-b2e3-1cad81dbe505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:49.745693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b86d88-4bf6-4106-9652-0f96c9c74064 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:49.752709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abec3f-1d45-4ea4-894f-24a55e081e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:49.754462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abec3f-1d45-4ea4-894f-24a55e081e10 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:49.754970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abec3f-1d45-4ea4-894f-24a55e081e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:49.755528 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abec3f-1d45-4ea4-894f-24a55e081e10 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:49.756469 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01abec3f-1d45-4ea4-894f-24a55e081e10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:49.759963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01abec3f-1d45-4ea4-894f-24a55e081e10 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:49.800510 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1065/2143] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:49 2022] POST /identity/v3/projects => generated 375 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:49.806456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abec3f-1d45-4ea4-894f-24a55e081e10 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:49.880176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675fa53-6100-40ff-b2e3-1cad81dbe505 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:49.898029 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abec3f-1d45-4ea4-894f-24a55e081e10 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:49.906381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ba52c-f861-4832-bbf9-d61efa16bc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:49.907555 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ba52c-f861-4832-bbf9-d61efa16bc21 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:49.908063 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ba52c-f861-4832-bbf9-d61efa16bc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:49.908507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ba52c-f861-4832-bbf9-d61efa16bc21 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:49.909473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ba52c-f861-4832-bbf9-d61efa16bc21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:49.912254 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ba52c-f861-4832-bbf9-d61efa16bc21 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:49.943354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1066/2144] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:49 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:49.949950 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ba52c-f861-4832-bbf9-d61efa16bc21 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:49.975976 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ba52c-f861-4832-bbf9-d61efa16bc21 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:49.984044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87339234-2f1b-4000-84f0-68120cb53fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:49.985046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87339234-2f1b-4000-84f0-68120cb53fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:49.985558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87339234-2f1b-4000-84f0-68120cb53fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:49.986325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87339234-2f1b-4000-84f0-68120cb53fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:49.987675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87339234-2f1b-4000-84f0-68120cb53fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:49.990275 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87339234-2f1b-4000-84f0-68120cb53fb5 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:50.009791 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1067/2145] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:50.015398 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87339234-2f1b-4000-84f0-68120cb53fb5 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:50.041092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87339234-2f1b-4000-84f0-68120cb53fb5 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:50.048203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2847a3-ed4b-4f9a-af20-a180dd0747e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:50.049758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2847a3-ed4b-4f9a-af20-a180dd0747e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:50.050047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2847a3-ed4b-4f9a-af20-a180dd0747e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:50.050406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2847a3-ed4b-4f9a-af20-a180dd0747e1 admin admin] PATH_INFO: `/v3/projects/8f1fcc06288f48e7b3660f907c8d9ff5/users/81c403619891468fbe8f4b808388d1f9/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:50.055193 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc7e4a-6b9a-457a-998d-c3be54cb0d05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:50.058951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc7e4a-6b9a-457a-998d-c3be54cb0d05 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:50.059367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc7e4a-6b9a-457a-998d-c3be54cb0d05 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:50.059752 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc7e4a-6b9a-457a-998d-c3be54cb0d05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:50.070869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2847a3-ed4b-4f9a-af20-a180dd0747e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f1fcc06288f48e7b3660f907c8d9ff5, user_id=81c403619891468fbe8f4b808388d1f9, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:50.074436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2847a3-ed4b-4f9a-af20-a180dd0747e1 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:50.083486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc7e4a-6b9a-457a-998d-c3be54cb0d05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:50.085603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc7e4a-6b9a-457a-998d-c3be54cb0d05 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:50.098195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1068/2146] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:50 2022] PUT /identity/v3/projects/8f1fcc06288f48e7b3660f907c8d9ff5/users/81c403619891468fbe8f4b808388d1f9/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:50.100813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2847a3-ed4b-4f9a-af20-a180dd0747e1 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:50.121740 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1079/2147] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:50.223013 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2847a3-ed4b-4f9a-af20-a180dd0747e1 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:50.231388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92590d-d680-498a-b04c-478bea18ee29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:50.232104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92590d-d680-498a-b04c-478bea18ee29 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:50.232388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92590d-d680-498a-b04c-478bea18ee29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:50.232668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92590d-d680-498a-b04c-478bea18ee29 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:50.233404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92590d-d680-498a-b04c-478bea18ee29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:50.235066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92590d-d680-498a-b04c-478bea18ee29 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:50.250755 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1069/2148] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:52:50.254748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc7e4a-6b9a-457a-998d-c3be54cb0d05 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:50.270530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc7e4a-6b9a-457a-998d-c3be54cb0d05 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:50.280219 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb592b9-a75e-4611-92d8-e650eb2cb666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:50.281921 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb592b9-a75e-4611-92d8-e650eb2cb666 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:50.282374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb592b9-a75e-4611-92d8-e650eb2cb666 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:50.283815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb592b9-a75e-4611-92d8-e650eb2cb666 admin admin] PATH_INFO: `/v3/projects/8f1fcc06288f48e7b3660f907c8d9ff5/users/81c403619891468fbe8f4b808388d1f9/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:50.287874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb592b9-a75e-4611-92d8-e650eb2cb666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f1fcc06288f48e7b3660f907c8d9ff5, user_id=81c403619891468fbe8f4b808388d1f9, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:50.292481 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb592b9-a75e-4611-92d8-e650eb2cb666 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:50.315012 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1080/2149] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:50 2022] PUT /identity/v3/projects/8f1fcc06288f48e7b3660f907c8d9ff5/users/81c403619891468fbe8f4b808388d1f9/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:50.321694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d5161-e4c8-41c1-8c32-8da7b3ff001c None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:50.322202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d5161-e4c8-41c1-8c32-8da7b3ff001c None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:50.322651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d5161-e4c8-41c1-8c32-8da7b3ff001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:50.374912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-cb0d5161-e4c8-41c1-8c32-8da7b3ff001c None None] MFA Rules not processed for user `81c403619891468fbe8f4b808388d1f9`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:50.376986 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-cb0d5161-e4c8-41c1-8c32-8da7b3ff001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:50.463143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb592b9-a75e-4611-92d8-e650eb2cb666 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:50.465972 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-fdb592b9-a75e-4611-92d8-e650eb2cb666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:50.554633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1070/2150] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:50.555843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d5161-e4c8-41c1-8c32-8da7b3ff001c None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:50.557967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-cb0d5161-e4c8-41c1-8c32-8da7b3ff001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:50.658192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d5161-e4c8-41c1-8c32-8da7b3ff001c None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:50.661904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb592b9-a75e-4611-92d8-e650eb2cb666 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:50.663304 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e9b5b-644e-49aa-9921-beedf249a848 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:50.664130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e9b5b-644e-49aa-9921-beedf249a848 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:50.664694 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e9b5b-644e-49aa-9921-beedf249a848 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:50.665110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e9b5b-644e-49aa-9921-beedf249a848 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:50.667328 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c35e9b5b-644e-49aa-9921-beedf249a848 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:50.669365 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:50.671188 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:50.671570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:50.671943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647 admin admin] PATH_INFO: `/v3/projects/c0e44b87f674490a9d95d2ae148a0e1a` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:50.673432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0e44b87f674490a9d95d2ae148a0e1a)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:50.677526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:50.741145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e9b5b-644e-49aa-9921-beedf249a848 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:50.743007 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e9b5b-644e-49aa-9921-beedf249a848 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:50.749806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0e44b87f674490a9d95d2ae148a0e1a', 'request_id': 'req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:50.757384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1071/2151] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:52:50.777782 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0e44b87f674490a9d95d2ae148a0e1a', 'request_id': 'req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:50.797376 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0e44b87f674490a9d95d2ae148a0e1a', 'request_id': 'req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:50.831942 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647 admin admin] The token cache is being invalidate because project c0e44b87f674490a9d95d2ae148a0e1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:50.832267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:50.832968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-8ec4f9a6-1d8d-4af6-97e9-768a67b4d647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:50.837880 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1081/2152] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:50 2022] DELETE /identity/v3/projects/c0e44b87f674490a9d95d2ae148a0e1a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:50.840128 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd4689-41dc-46bf-a8ec-6bfbf00fd320 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:50.840412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd4689-41dc-46bf-a8ec-6bfbf00fd320 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:50.840697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd4689-41dc-46bf-a8ec-6bfbf00fd320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:50.865105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-61fd4689-41dc-46bf-a8ec-6bfbf00fd320 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:50.866615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-61fd4689-41dc-46bf-a8ec-6bfbf00fd320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:50.999911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1072/2153] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:51.010953 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7a9c4-3066-4d99-9a3f-b2a91f4f327a None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:51.011255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7a9c4-3066-4d99-9a3f-b2a91f4f327a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:51.011580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7a9c4-3066-4d99-9a3f-b2a91f4f327a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:51.018102 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd4689-41dc-46bf-a8ec-6bfbf00fd320 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:51.019616 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-61fd4689-41dc-46bf-a8ec-6bfbf00fd320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:51.044868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd4689-41dc-46bf-a8ec-6bfbf00fd320 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:51.055144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:51.056012 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:51.056305 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:51.056584 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] PATH_INFO: `/v3/projects/6b17a044889b481cad63732c38a1d89a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:51.057866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b17a044889b481cad63732c38a1d89a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:51.059835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:51.079800 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-42c7a9c4-3066-4d99-9a3f-b2a91f4f327a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:51.083033 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-42c7a9c4-3066-4d99-9a3f-b2a91f4f327a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:51.111948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1082/2154] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:51.115204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7a9c4-3066-4d99-9a3f-b2a91f4f327a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:51.133236 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7a9c4-3066-4d99-9a3f-b2a91f4f327a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:51.179937 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0935ab-7adc-4f75-bce1-c70e58b6c103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:51.180882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0935ab-7adc-4f75-bce1-c70e58b6c103 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:51.181170 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0935ab-7adc-4f75-bce1-c70e58b6c103 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:51.181532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0935ab-7adc-4f75-bce1-c70e58b6c103 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:51.182271 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0935ab-7adc-4f75-bce1-c70e58b6c103 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:51.182586 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0935ab-7adc-4f75-bce1-c70e58b6c103 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:51.184394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0935ab-7adc-4f75-bce1-c70e58b6c103 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:51.196901 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b17a044889b481cad63732c38a1d89a', 'request_id': 'req-2f4dbd83-35a9-4c7e-9208-bb56048490a6'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:51.204753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b17a044889b481cad63732c38a1d89a', 'request_id': 'req-2f4dbd83-35a9-4c7e-9208-bb56048490a6'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:51.230539 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1083/2155] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:51.232997 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0935ab-7adc-4f75-bce1-c70e58b6c103 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:51.233122 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b17a044889b481cad63732c38a1d89a', 'request_id': 'req-2f4dbd83-35a9-4c7e-9208-bb56048490a6'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:51.261398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0935ab-7adc-4f75-bce1-c70e58b6c103 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:51.275689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] The token cache is being invalidate because project 6b17a044889b481cad63732c38a1d89a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:51.275689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:51.276909 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:51.280974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1073/2156] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:51 2022] DELETE /identity/v3/projects/6b17a044889b481cad63732c38a1d89a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:51.281986 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:51.283458 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:51.431692 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03637bf7-f1b3-4891-8033-811138b06e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:51.432381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03637bf7-f1b3-4891-8033-811138b06e17 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:51.432661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03637bf7-f1b3-4891-8033-811138b06e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:51.433107 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03637bf7-f1b3-4891-8033-811138b06e17 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:51.433965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03637bf7-f1b3-4891-8033-811138b06e17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:51.441024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03637bf7-f1b3-4891-8033-811138b06e17 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:51.488352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4dbd83-35a9-4c7e-9208-bb56048490a6 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:51.497391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1084/2157] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:51 2022] POST /identity/v3/projects => generated 371 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:51.498872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03637bf7-f1b3-4891-8033-811138b06e17 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:51.503679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-03637bf7-f1b3-4891-8033-811138b06e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:51.504312 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72eae6f-96f4-4c08-9a55-b6a163e45523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:51.505487 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72eae6f-96f4-4c08-9a55-b6a163e45523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:51.505961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72eae6f-96f4-4c08-9a55-b6a163e45523 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:51.506430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72eae6f-96f4-4c08-9a55-b6a163e45523 admin admin] PATH_INFO: `/v3/users/89db057e13a346769ca9cdf54acd31c6` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:51.507988 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72eae6f-96f4-4c08-9a55-b6a163e45523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89db057e13a346769ca9cdf54acd31c6)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:51.510448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72eae6f-96f4-4c08-9a55-b6a163e45523 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:51.601462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f72eae6f-96f4-4c08-9a55-b6a163e45523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89db057e13a346769ca9cdf54acd31c6'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:51.628954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f72eae6f-96f4-4c08-9a55-b6a163e45523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89db057e13a346769ca9cdf54acd31c6'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:51.630538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f72eae6f-96f4-4c08-9a55-b6a163e45523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89db057e13a346769ca9cdf54acd31c6'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:51.634325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f72eae6f-96f4-4c08-9a55-b6a163e45523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89db057e13a346769ca9cdf54acd31c6'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:51.652390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f72eae6f-96f4-4c08-9a55-b6a163e45523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89db057e13a346769ca9cdf54acd31c6'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:51.667926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03637bf7-f1b3-4891-8033-811138b06e17 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:51.680709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1074/2158] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:51 2022] DELETE /identity/v3/users/89db057e13a346769ca9cdf54acd31c6 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:51.688012 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a20e0-5126-45a9-8323-3d70a0451de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:51.688842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a20e0-5126-45a9-8323-3d70a0451de9 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:51.688971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a20e0-5126-45a9-8323-3d70a0451de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:51.689246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a20e0-5126-45a9-8323-3d70a0451de9 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:51.689937 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a20e0-5126-45a9-8323-3d70a0451de9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:51.698246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a20e0-5126-45a9-8323-3d70a0451de9 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:51.743453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1085/2159] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:51 2022] POST /identity/v3/users => generated 286 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:51.748251 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72eae6f-96f4-4c08-9a55-b6a163e45523 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:51.749914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f72eae6f-96f4-4c08-9a55-b6a163e45523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:51.897102 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72eae6f-96f4-4c08-9a55-b6a163e45523 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:51.903475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb5262-1eef-4cf9-b46a-fd7b25dd1d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:51.904231 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb5262-1eef-4cf9-b46a-fd7b25dd1d62 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:51.904516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb5262-1eef-4cf9-b46a-fd7b25dd1d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:51.904795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb5262-1eef-4cf9-b46a-fd7b25dd1d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:51.905752 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb5262-1eef-4cf9-b46a-fd7b25dd1d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:51.907606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb5262-1eef-4cf9-b46a-fd7b25dd1d62 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:51.919879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1075/2160] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:51.923542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a20e0-5126-45a9-8323-3d70a0451de9 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:51.954844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a20e0-5126-45a9-8323-3d70a0451de9 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:51.965664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b52b3-4fb2-4e59-905b-ab7bfb016b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:51.967176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b52b3-4fb2-4e59-905b-ab7bfb016b29 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:51.968216 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b52b3-4fb2-4e59-905b-ab7bfb016b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:51.968842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b52b3-4fb2-4e59-905b-ab7bfb016b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:51.971760 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b52b3-4fb2-4e59-905b-ab7bfb016b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:51.973645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b52b3-4fb2-4e59-905b-ab7bfb016b29 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:51.991894 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1086/2161] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:51.996511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb5262-1eef-4cf9-b46a-fd7b25dd1d62 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:52.008716 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb5262-1eef-4cf9-b46a-fd7b25dd1d62 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:52.025516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5a9ca-5bd6-48b5-86eb-d234f196b5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:52.026527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5a9ca-5bd6-48b5-86eb-d234f196b5d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:52.026845 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5a9ca-5bd6-48b5-86eb-d234f196b5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:52.027122 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5a9ca-5bd6-48b5-86eb-d234f196b5d7 admin admin] PATH_INFO: `/v3/projects/4e3dea64bae54cc1bae11a22214c60fa/users/762d8a40f938401b81698542fb6714f5/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:52.053197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5a9ca-5bd6-48b5-86eb-d234f196b5d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e3dea64bae54cc1bae11a22214c60fa, user_id=762d8a40f938401b81698542fb6714f5, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:52.057717 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5a9ca-5bd6-48b5-86eb-d234f196b5d7 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:52.083240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1076/2162] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:51 2022] PUT /identity/v3/projects/4e3dea64bae54cc1bae11a22214c60fa/users/762d8a40f938401b81698542fb6714f5/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:52.094315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10009b6-5d91-46e7-bd8d-1d9b022b8c16 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:52.094606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10009b6-5d91-46e7-bd8d-1d9b022b8c16 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:52.094944 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10009b6-5d91-46e7-bd8d-1d9b022b8c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:52.133679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-f10009b6-5d91-46e7-bd8d-1d9b022b8c16 None None] MFA Rules not processed for user `762d8a40f938401b81698542fb6714f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:52.134473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f10009b6-5d91-46e7-bd8d-1d9b022b8c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:52.233696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1087/2163] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:52.237738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5a9ca-5bd6-48b5-86eb-d234f196b5d7 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:52.370626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5a9ca-5bd6-48b5-86eb-d234f196b5d7 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:52.400981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3838bed-b186-4cdf-80fc-c1079f03d631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:52.401993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3838bed-b186-4cdf-80fc-c1079f03d631 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:52.402285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3838bed-b186-4cdf-80fc-c1079f03d631 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:52.402645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3838bed-b186-4cdf-80fc-c1079f03d631 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:52.410462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3838bed-b186-4cdf-80fc-c1079f03d631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:52.413589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3838bed-b186-4cdf-80fc-c1079f03d631 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:52.496885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1077/2164] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:52 2022] POST /identity/v3/projects => generated 373 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:52.497249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10009b6-5d91-46e7-bd8d-1d9b022b8c16 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:52.535780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10009b6-5d91-46e7-bd8d-1d9b022b8c16 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:52.724573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87468557-dcb7-45e5-b073-a1bf8dbc3fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:52.725254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87468557-dcb7-45e5-b073-a1bf8dbc3fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:52.725702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87468557-dcb7-45e5-b073-a1bf8dbc3fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:52.726177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87468557-dcb7-45e5-b073-a1bf8dbc3fa1 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:52.731328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87468557-dcb7-45e5-b073-a1bf8dbc3fa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:52.733872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87468557-dcb7-45e5-b073-a1bf8dbc3fa1 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:52.793369 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3838bed-b186-4cdf-80fc-c1079f03d631 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:52.794015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1088/2165] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:52 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:52.799245 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c3838bed-b186-4cdf-80fc-c1079f03d631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:52.799428 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87468557-dcb7-45e5-b073-a1bf8dbc3fa1 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:52.821975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87468557-dcb7-45e5-b073-a1bf8dbc3fa1 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:52.847698 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37743ad5-eddf-4fc1-96d2-365348d8891a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:52.850097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37743ad5-eddf-4fc1-96d2-365348d8891a admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:52.851170 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37743ad5-eddf-4fc1-96d2-365348d8891a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:52.851635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37743ad5-eddf-4fc1-96d2-365348d8891a admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:52.852574 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37743ad5-eddf-4fc1-96d2-365348d8891a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:52.855832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37743ad5-eddf-4fc1-96d2-365348d8891a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:52.892437 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1089/2166] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:52.897807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37743ad5-eddf-4fc1-96d2-365348d8891a admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:52.927859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3838bed-b186-4cdf-80fc-c1079f03d631 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:52.931916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877b824-1354-441c-9252-a50e7e0bbefc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:52.932660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877b824-1354-441c-9252-a50e7e0bbefc service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:52.932775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37743ad5-eddf-4fc1-96d2-365348d8891a admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:52.933212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877b824-1354-441c-9252-a50e7e0bbefc service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:52.933544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877b824-1354-441c-9252-a50e7e0bbefc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:52.936122 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f877b824-1354-441c-9252-a50e7e0bbefc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:52.945277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac82bcd-d13a-47fb-9ca3-1cb93f5e9cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:52.946528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac82bcd-d13a-47fb-9ca3-1cb93f5e9cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:52.946866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac82bcd-d13a-47fb-9ca3-1cb93f5e9cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:52.947148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac82bcd-d13a-47fb-9ca3-1cb93f5e9cb4 admin admin] PATH_INFO: `/v3/projects/abcb82a62c34445980fce3b38fa81c6a/users/9bfc987f64964d9e9cbdd6be35ef7a5b/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:52.966497 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac82bcd-d13a-47fb-9ca3-1cb93f5e9cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abcb82a62c34445980fce3b38fa81c6a, user_id=9bfc987f64964d9e9cbdd6be35ef7a5b, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:52.969810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac82bcd-d13a-47fb-9ca3-1cb93f5e9cb4 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:52.998940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1090/2167] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:52 2022] PUT /identity/v3/projects/abcb82a62c34445980fce3b38fa81c6a/users/9bfc987f64964d9e9cbdd6be35ef7a5b/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:53.004910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac82bcd-d13a-47fb-9ca3-1cb93f5e9cb4 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:53.027293 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877b824-1354-441c-9252-a50e7e0bbefc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:53.032480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877b824-1354-441c-9252-a50e7e0bbefc service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:53.057518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1078/2168] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:53.194437 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac82bcd-d13a-47fb-9ca3-1cb93f5e9cb4 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:53.206946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48015a2-3b6e-4d69-90d9-357e18f16432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:53.207973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48015a2-3b6e-4d69-90d9-357e18f16432 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:53.209097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48015a2-3b6e-4d69-90d9-357e18f16432 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:53.209567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48015a2-3b6e-4d69-90d9-357e18f16432 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:53.210486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48015a2-3b6e-4d69-90d9-357e18f16432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:53.213160 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48015a2-3b6e-4d69-90d9-357e18f16432 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:53.228726 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877b824-1354-441c-9252-a50e7e0bbefc service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:53.230592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f877b824-1354-441c-9252-a50e7e0bbefc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:53.234911 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1091/2169] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:52:53.238809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48015a2-3b6e-4d69-90d9-357e18f16432 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:53.251177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877b824-1354-441c-9252-a50e7e0bbefc service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:53.261692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c86c2-4047-4f0c-8508-3ec8644f9c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:53.267307 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48015a2-3b6e-4d69-90d9-357e18f16432 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:53.269060 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c86c2-4047-4f0c-8508-3ec8644f9c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:53.269363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c86c2-4047-4f0c-8508-3ec8644f9c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:53.269651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c86c2-4047-4f0c-8508-3ec8644f9c0d admin admin] PATH_INFO: `/v3/projects/d1d4fbc895b241abb9d8a895756fe9cf` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:53.273073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c86c2-4047-4f0c-8508-3ec8644f9c0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d4fbc895b241abb9d8a895756fe9cf)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:53.281256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c86c2-4047-4f0c-8508-3ec8644f9c0d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:53.284714 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f11895-0c00-42d5-b0b7-b05bb8c458a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:53.286882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f11895-0c00-42d5-b0b7-b05bb8c458a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:53.287543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f11895-0c00-42d5-b0b7-b05bb8c458a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:53.287832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f11895-0c00-42d5-b0b7-b05bb8c458a2 admin admin] PATH_INFO: `/v3/projects/abcb82a62c34445980fce3b38fa81c6a/users/9bfc987f64964d9e9cbdd6be35ef7a5b/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:53.290575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f11895-0c00-42d5-b0b7-b05bb8c458a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abcb82a62c34445980fce3b38fa81c6a, user_id=9bfc987f64964d9e9cbdd6be35ef7a5b, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:53.293609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f11895-0c00-42d5-b0b7-b05bb8c458a2 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:53.326154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1092/2170] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:53 2022] PUT /identity/v3/projects/abcb82a62c34445980fce3b38fa81c6a/users/9bfc987f64964d9e9cbdd6be35ef7a5b/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:53.337946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1e639-04f1-4f93-935a-b7b83250937f None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:53.338403 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1e639-04f1-4f93-935a-b7b83250937f None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:53.338996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1e639-04f1-4f93-935a-b7b83250937f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:53.383421 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-995c86c2-4047-4f0c-8508-3ec8644f9c0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d4fbc895b241abb9d8a895756fe9cf', 'request_id': 'req-995c86c2-4047-4f0c-8508-3ec8644f9c0d'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:53.392206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-995c86c2-4047-4f0c-8508-3ec8644f9c0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d4fbc895b241abb9d8a895756fe9cf', 'request_id': 'req-995c86c2-4047-4f0c-8508-3ec8644f9c0d'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:53.397577 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-92a1e639-04f1-4f93-935a-b7b83250937f None None] MFA Rules not processed for user `9bfc987f64964d9e9cbdd6be35ef7a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:53.399003 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-92a1e639-04f1-4f93-935a-b7b83250937f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:53.423569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-995c86c2-4047-4f0c-8508-3ec8644f9c0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d4fbc895b241abb9d8a895756fe9cf', 'request_id': 'req-995c86c2-4047-4f0c-8508-3ec8644f9c0d'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:53.438610 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-995c86c2-4047-4f0c-8508-3ec8644f9c0d admin admin] The token cache is being invalidate because project d1d4fbc895b241abb9d8a895756fe9cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:53.439259 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-995c86c2-4047-4f0c-8508-3ec8644f9c0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:53.440092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-995c86c2-4047-4f0c-8508-3ec8644f9c0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:53.442750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1079/2171] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:53 2022] DELETE /identity/v3/projects/d1d4fbc895b241abb9d8a895756fe9cf => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:53.447860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83045f-7e07-484b-8144-e9c2a3b77995 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:53.449447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83045f-7e07-484b-8144-e9c2a3b77995 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:53.452010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83045f-7e07-484b-8144-e9c2a3b77995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:53.491287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-dc83045f-7e07-484b-8144-e9c2a3b77995 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:53.493835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-dc83045f-7e07-484b-8144-e9c2a3b77995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:53.545479 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1093/2172] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:53.558090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1e639-04f1-4f93-935a-b7b83250937f None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:53.560002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-92a1e639-04f1-4f93-935a-b7b83250937f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:53.606389 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1080/2173] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:53.608792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83c649-c051-4a89-b22d-871e3ba1c454 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:53.609318 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83c649-c051-4a89-b22d-871e3ba1c454 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:53.609852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83c649-c051-4a89-b22d-871e3ba1c454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:53.658990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-0b83c649-c051-4a89-b22d-871e3ba1c454 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:53.660934 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0b83c649-c051-4a89-b22d-871e3ba1c454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:53.678366 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1081/2174] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:53.679611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83c649-c051-4a89-b22d-871e3ba1c454 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:53.680910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0b83c649-c051-4a89-b22d-871e3ba1c454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:53.682993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1e639-04f1-4f93-935a-b7b83250937f None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:53.688466 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981cebd-1006-4904-a2ff-a72b7535c784 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:53.689461 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981cebd-1006-4904-a2ff-a72b7535c784 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:53.690384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981cebd-1006-4904-a2ff-a72b7535c784 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:53.690869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981cebd-1006-4904-a2ff-a72b7535c784 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:53.709221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981cebd-1006-4904-a2ff-a72b7535c784 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:53.710156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83c649-c051-4a89-b22d-871e3ba1c454 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:53.712225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981cebd-1006-4904-a2ff-a72b7535c784 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:53.718729 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4328f1-22c7-4ee9-aad3-ea977b76765c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:53.719754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4328f1-22c7-4ee9-aad3-ea977b76765c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:53.720125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4328f1-22c7-4ee9-aad3-ea977b76765c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:53.721465 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4328f1-22c7-4ee9-aad3-ea977b76765c admin admin] PATH_INFO: `/v3/users/2b6403f474fa468498fe0a5e753c1203` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:53.723120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4328f1-22c7-4ee9-aad3-ea977b76765c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b6403f474fa468498fe0a5e753c1203)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:53.727055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4328f1-22c7-4ee9-aad3-ea977b76765c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:53.737874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1094/2175] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:52:53.741272 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981cebd-1006-4904-a2ff-a72b7535c784 service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:53.780446 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981cebd-1006-4904-a2ff-a72b7535c784 service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:53.799917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43ff8c-36a9-457d-814b-94b815418b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:53.802271 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43ff8c-36a9-457d-814b-94b815418b21 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:53.803009 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43ff8c-36a9-457d-814b-94b815418b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:53.803449 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43ff8c-36a9-457d-814b-94b815418b21 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:53.804187 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43ff8c-36a9-457d-814b-94b815418b21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:53.804663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43ff8c-36a9-457d-814b-94b815418b21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:53.806947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43ff8c-36a9-457d-814b-94b815418b21 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:53.834976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1095/2176] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:53.842208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43ff8c-36a9-457d-814b-94b815418b21 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:53.851063 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3a4328f1-22c7-4ee9-aad3-ea977b76765c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b6403f474fa468498fe0a5e753c1203'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:53.872137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43ff8c-36a9-457d-814b-94b815418b21 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:53.876376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3a4328f1-22c7-4ee9-aad3-ea977b76765c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b6403f474fa468498fe0a5e753c1203'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:53.877192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3a4328f1-22c7-4ee9-aad3-ea977b76765c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b6403f474fa468498fe0a5e753c1203'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:53.878012 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3a4328f1-22c7-4ee9-aad3-ea977b76765c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b6403f474fa468498fe0a5e753c1203'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:53.883920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3669e3-23c3-4ea7-9e27-7d8295458a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:53.884732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3669e3-23c3-4ea7-9e27-7d8295458a72 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:53.885015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3669e3-23c3-4ea7-9e27-7d8295458a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:53.885292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3669e3-23c3-4ea7-9e27-7d8295458a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:53.887879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3a4328f1-22c7-4ee9-aad3-ea977b76765c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b6403f474fa468498fe0a5e753c1203'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:53.888093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3669e3-23c3-4ea7-9e27-7d8295458a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:53.892420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3669e3-23c3-4ea7-9e27-7d8295458a72 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:53.911683 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1082/2177] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:53 2022] DELETE /identity/v3/users/2b6403f474fa468498fe0a5e753c1203 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:53.944182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1096/2178] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:53 2022] POST /identity/v3/projects => generated 357 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:53.948066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4328f1-22c7-4ee9-aad3-ea977b76765c admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:53.949176 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-3a4328f1-22c7-4ee9-aad3-ea977b76765c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:54.107853 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4328f1-22c7-4ee9-aad3-ea977b76765c admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:54.180749 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248565ae-6ed5-4e41-a4ff-b67dceb852c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:54.181427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248565ae-6ed5-4e41-a4ff-b67dceb852c6 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:54.182125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248565ae-6ed5-4e41-a4ff-b67dceb852c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:54.182671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248565ae-6ed5-4e41-a4ff-b67dceb852c6 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:54.183623 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248565ae-6ed5-4e41-a4ff-b67dceb852c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:54.185614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248565ae-6ed5-4e41-a4ff-b67dceb852c6 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:54.219069 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3669e3-23c3-4ea7-9e27-7d8295458a72 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:54.219999 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1083/2179] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:53 2022] POST /identity/v3/users => generated 279 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:54.224087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ba3669e3-23c3-4ea7-9e27-7d8295458a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:54.224202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248565ae-6ed5-4e41-a4ff-b67dceb852c6 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:54.236653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248565ae-6ed5-4e41-a4ff-b67dceb852c6 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:54.250212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dd3e3-12e4-4283-89ec-ad62820bdd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:54.250993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dd3e3-12e4-4283-89ec-ad62820bdd24 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:54.251273 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dd3e3-12e4-4283-89ec-ad62820bdd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:54.252150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dd3e3-12e4-4283-89ec-ad62820bdd24 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:54.252271 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dd3e3-12e4-4283-89ec-ad62820bdd24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:54.256325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dd3e3-12e4-4283-89ec-ad62820bdd24 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:54.289944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1084/2180] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:54.291465 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dd3e3-12e4-4283-89ec-ad62820bdd24 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:54.321474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dd3e3-12e4-4283-89ec-ad62820bdd24 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:54.352998 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3669e3-23c3-4ea7-9e27-7d8295458a72 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:54.353991 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566df11c-550f-42f7-b7c4-cffb2f974bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:54.354760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566df11c-550f-42f7-b7c4-cffb2f974bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:54.355136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566df11c-550f-42f7-b7c4-cffb2f974bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:54.355608 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566df11c-550f-42f7-b7c4-cffb2f974bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:54.356272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566df11c-550f-42f7-b7c4-cffb2f974bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:54.358149 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566df11c-550f-42f7-b7c4-cffb2f974bb2 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:54.364276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca66b5-1463-4d9f-9c93-dfd6bb5d898f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:54.365025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca66b5-1463-4d9f-9c93-dfd6bb5d898f service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:54.365778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca66b5-1463-4d9f-9c93-dfd6bb5d898f service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:54.366147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca66b5-1463-4d9f-9c93-dfd6bb5d898f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:54.369020 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-caca66b5-1463-4d9f-9c93-dfd6bb5d898f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:54.377467 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1085/2181] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:54.377467 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566df11c-550f-42f7-b7c4-cffb2f974bb2 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:54.378903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-566df11c-550f-42f7-b7c4-cffb2f974bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:54.396275 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566df11c-550f-42f7-b7c4-cffb2f974bb2 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:54.401671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:54.402733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:54.403113 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:54.403462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] PATH_INFO: `/v3/projects/0edcbbe2440f4666b68289635de4adea` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:54.405475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0edcbbe2440f4666b68289635de4adea)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:54.408010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:54.495700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0edcbbe2440f4666b68289635de4adea', 'request_id': 'req-f85202c7-43a4-4371-bb23-a2dc7294657d'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:54.508980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0edcbbe2440f4666b68289635de4adea', 'request_id': 'req-f85202c7-43a4-4371-bb23-a2dc7294657d'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:54.536894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0edcbbe2440f4666b68289635de4adea', 'request_id': 'req-f85202c7-43a4-4371-bb23-a2dc7294657d'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:54.537410 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca66b5-1463-4d9f-9c93-dfd6bb5d898f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:54.539191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca66b5-1463-4d9f-9c93-dfd6bb5d898f service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:54.553232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] The token cache is being invalidate because project 0edcbbe2440f4666b68289635de4adea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:54.553469 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:54.554033 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:54.558970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1086/2182] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:54 2022] DELETE /identity/v3/projects/0edcbbe2440f4666b68289635de4adea => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:54.558970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:54.560381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:54.568900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1097/2183] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:54 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:52:54.570356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca66b5-1463-4d9f-9c93-dfd6bb5d898f service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:54.585542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca66b5-1463-4d9f-9c93-dfd6bb5d898f service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:54.663259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85eca7-6501-4364-906e-87ebc5a16f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:54.664232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85eca7-6501-4364-906e-87ebc5a16f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:54.664605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85eca7-6501-4364-906e-87ebc5a16f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:54.665114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85eca7-6501-4364-906e-87ebc5a16f1f admin admin] PATH_INFO: `/v3/users/1bbd1285beb34cd28e0b51f6ea059596` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:54.666540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85eca7-6501-4364-906e-87ebc5a16f1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bbd1285beb34cd28e0b51f6ea059596)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:54.668779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85eca7-6501-4364-906e-87ebc5a16f1f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:54.731067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85202c7-43a4-4371-bb23-a2dc7294657d admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:54.738920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50d2be-7720-4a91-9b72-820574b81bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:54.739870 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50d2be-7720-4a91-9b72-820574b81bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:54.740751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50d2be-7720-4a91-9b72-820574b81bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:54.741271 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50d2be-7720-4a91-9b72-820574b81bdc admin admin] PATH_INFO: `/v3/projects/851dcca13cee48a19327e7bada45af31/users/f56effc6e229462882283f5bbf0bd209/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:54.760698 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50d2be-7720-4a91-9b72-820574b81bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=851dcca13cee48a19327e7bada45af31, user_id=f56effc6e229462882283f5bbf0bd209, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:54.764499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50d2be-7720-4a91-9b72-820574b81bdc admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:54.787472 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1087/2184] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:54 2022] PUT /identity/v3/projects/851dcca13cee48a19327e7bada45af31/users/f56effc6e229462882283f5bbf0bd209/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:54.789383 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50d2be-7720-4a91-9b72-820574b81bdc admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:54.797844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-db85eca7-6501-4364-906e-87ebc5a16f1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bbd1285beb34cd28e0b51f6ea059596'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:54.802868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-fe50d2be-7720-4a91-9b72-820574b81bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:54.819489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-db85eca7-6501-4364-906e-87ebc5a16f1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bbd1285beb34cd28e0b51f6ea059596'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:54.820506 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-db85eca7-6501-4364-906e-87ebc5a16f1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bbd1285beb34cd28e0b51f6ea059596'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:54.821278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-db85eca7-6501-4364-906e-87ebc5a16f1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bbd1285beb34cd28e0b51f6ea059596'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:54.830133 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-db85eca7-6501-4364-906e-87ebc5a16f1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bbd1285beb34cd28e0b51f6ea059596'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:54.863812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1098/2185] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:54 2022] DELETE /identity/v3/users/1bbd1285beb34cd28e0b51f6ea059596 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:54.865892 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fa09f-a45d-4539-936f-de340c440f0a None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:54.866351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fa09f-a45d-4539-936f-de340c440f0a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:54.875603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fa09f-a45d-4539-936f-de340c440f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:54.925766 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-789fa09f-a45d-4539-936f-de340c440f0a None None] MFA Rules not processed for user `f56effc6e229462882283f5bbf0bd209`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:54.927964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-789fa09f-a45d-4539-936f-de340c440f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:54.943951 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50d2be-7720-4a91-9b72-820574b81bdc admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:55.024602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1099/2186] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:54 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:55.032924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789fa09f-a45d-4539-936f-de340c440f0a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:55.034748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-789fa09f-a45d-4539-936f-de340c440f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:55.095767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c49d8-2cce-414f-9c57-360c8938a2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:55.096651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c49d8-2cce-414f-9c57-360c8938a2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:55.096941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c49d8-2cce-414f-9c57-360c8938a2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:55.097225 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c49d8-2cce-414f-9c57-360c8938a2b7 admin admin] PATH_INFO: `/v3/users/762d8a40f938401b81698542fb6714f5` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:55.100045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c49d8-2cce-414f-9c57-360c8938a2b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=762d8a40f938401b81698542fb6714f5)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:55.102031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c49d8-2cce-414f-9c57-360c8938a2b7 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:55.130671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789fa09f-a45d-4539-936f-de340c440f0a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:55.148696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69abff-2a63-4d1d-b4c5-f06e69ccc5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:55.149393 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69abff-2a63-4d1d-b4c5-f06e69ccc5f5 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:55.149678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69abff-2a63-4d1d-b4c5-f06e69ccc5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:55.149956 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69abff-2a63-4d1d-b4c5-f06e69ccc5f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:55.156117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69abff-2a63-4d1d-b4c5-f06e69ccc5f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:55.156117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69abff-2a63-4d1d-b4c5-f06e69ccc5f5 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:55.188494 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1100/2187] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:55 2022] POST /identity/v3/projects => generated 357 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:55.189900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69abff-2a63-4d1d-b4c5-f06e69ccc5f5 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:55.192888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-4c69abff-2a63-4d1d-b4c5-f06e69ccc5f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:55.202216 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e27c49d8-2cce-414f-9c57-360c8938a2b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '762d8a40f938401b81698542fb6714f5'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:55.217692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e27c49d8-2cce-414f-9c57-360c8938a2b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '762d8a40f938401b81698542fb6714f5'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:55.218491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e27c49d8-2cce-414f-9c57-360c8938a2b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '762d8a40f938401b81698542fb6714f5'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:55.219160 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e27c49d8-2cce-414f-9c57-360c8938a2b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '762d8a40f938401b81698542fb6714f5'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:55.228027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e27c49d8-2cce-414f-9c57-360c8938a2b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '762d8a40f938401b81698542fb6714f5'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:55.244339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1088/2188] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:54 2022] DELETE /identity/v3/users/762d8a40f938401b81698542fb6714f5 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:55.245250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c49d8-2cce-414f-9c57-360c8938a2b7 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:55.246737 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-e27c49d8-2cce-414f-9c57-360c8938a2b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:55.272969 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69abff-2a63-4d1d-b4c5-f06e69ccc5f5 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:55.347249 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c49d8-2cce-414f-9c57-360c8938a2b7 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:55.352669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795e449-70ea-44ed-b802-bd676a921f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:55.353582 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795e449-70ea-44ed-b802-bd676a921f99 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:55.354373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795e449-70ea-44ed-b802-bd676a921f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:55.354806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795e449-70ea-44ed-b802-bd676a921f99 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:55.355969 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795e449-70ea-44ed-b802-bd676a921f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:55.359565 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795e449-70ea-44ed-b802-bd676a921f99 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:55.370540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:55.371835 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:55.372325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:55.372770 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] PATH_INFO: `/v3/projects/e464534e84de4491b568c6716574a689` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:55.374684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e464534e84de4491b568c6716574a689)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:55.377086 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:55.389390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1089/2189] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:55 2022] POST /identity/v3/users => generated 285 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:55.389390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c166e9e-bc90-40e5-8e2c-73740b371cc8 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:55.389390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c166e9e-bc90-40e5-8e2c-73740b371cc8 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:55.389390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c166e9e-bc90-40e5-8e2c-73740b371cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:55.408875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5c166e9e-bc90-40e5-8e2c-73740b371cc8 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:55.410239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5c166e9e-bc90-40e5-8e2c-73740b371cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:55.419080 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1090/2190] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:55.420272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c166e9e-bc90-40e5-8e2c-73740b371cc8 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:55.429799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c166e9e-bc90-40e5-8e2c-73740b371cc8 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:55.436730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7de337-a6ed-4553-8435-3210e53d0f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:55.437415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7de337-a6ed-4553-8435-3210e53d0f06 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:55.437691 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7de337-a6ed-4553-8435-3210e53d0f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:55.437969 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7de337-a6ed-4553-8435-3210e53d0f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:55.438793 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7de337-a6ed-4553-8435-3210e53d0f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:55.440492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7de337-a6ed-4553-8435-3210e53d0f06 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:55.455176 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1091/2191] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:55.456135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7de337-a6ed-4553-8435-3210e53d0f06 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:55.458068 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-dd7de337-a6ed-4553-8435-3210e53d0f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:55.461932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e464534e84de4491b568c6716574a689', 'request_id': 'req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:55.468900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e464534e84de4491b568c6716574a689', 'request_id': 'req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:55.479290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e464534e84de4491b568c6716574a689', 'request_id': 'req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:55.495689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] The token cache is being invalidate because project e464534e84de4491b568c6716574a689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:55.496125 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:55.496868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:55.498831 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1101/2192] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:55 2022] DELETE /identity/v3/projects/e464534e84de4491b568c6716574a689 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:55.499518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:55.500898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:55.537592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7de337-a6ed-4553-8435-3210e53d0f06 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:55.568245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9c760-ad1e-4c0f-af0b-368e51d246dc admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:55.573746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fd737-1aa6-488c-9793-caddad32efc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:55.574738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fd737-1aa6-488c-9793-caddad32efc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:55.575037 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fd737-1aa6-488c-9793-caddad32efc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:55.575316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fd737-1aa6-488c-9793-caddad32efc2 admin admin] PATH_INFO: `/v3/projects/c241599785234c62af723fe561c669c6/users/f417d7c9ad294948b2774846f9a1d239/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:55.592596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8c7dd-00e8-4755-bc48-0f9d55c042f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:55.592778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fd737-1aa6-488c-9793-caddad32efc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c241599785234c62af723fe561c669c6, user_id=f417d7c9ad294948b2774846f9a1d239, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:55.592969 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8c7dd-00e8-4755-bc48-0f9d55c042f0 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:55.593356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8c7dd-00e8-4755-bc48-0f9d55c042f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:55.593734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8c7dd-00e8-4755-bc48-0f9d55c042f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:55.595329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fd737-1aa6-488c-9793-caddad32efc2 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:55.596230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f2e8c7dd-00e8-4755-bc48-0f9d55c042f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:55.613738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8c7dd-00e8-4755-bc48-0f9d55c042f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:55.615679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8c7dd-00e8-4755-bc48-0f9d55c042f0 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:55.616003 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1102/2193] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:55 2022] PUT /identity/v3/projects/c241599785234c62af723fe561c669c6/users/f417d7c9ad294948b2774846f9a1d239/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:55.617484 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3e2a7-f5a6-463c-9a59-1e3a4d20c5cc None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:55.617777 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3e2a7-f5a6-463c-9a59-1e3a4d20c5cc None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:55.618060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3e2a7-f5a6-463c-9a59-1e3a4d20c5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:55.627307 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1092/2194] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:55.628643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8c7dd-00e8-4755-bc48-0f9d55c042f0 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:55.640038 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8c7dd-00e8-4755-bc48-0f9d55c042f0 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:55.646055 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-4cd3e2a7-f5a6-463c-9a59-1e3a4d20c5cc None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:55.648430 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-4cd3e2a7-f5a6-463c-9a59-1e3a4d20c5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:55.738734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e7dc2-4fc6-4a6b-8161-6d985c064252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:55.739339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e7dc2-4fc6-4a6b-8161-6d985c064252 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:55.739629 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e7dc2-4fc6-4a6b-8161-6d985c064252 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:55.739906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e7dc2-4fc6-4a6b-8161-6d985c064252 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:55.741482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1103/2195] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:55.741604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e7dc2-4fc6-4a6b-8161-6d985c064252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:55.743326 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e7dc2-4fc6-4a6b-8161-6d985c064252 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:55.754252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44961ca-bb06-4811-a446-a2835da2c92c None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:55.754573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44961ca-bb06-4811-a446-a2835da2c92c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:55.754887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44961ca-bb06-4811-a446-a2835da2c92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:55.758769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1093/2196] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:55.760793 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e7dc2-4fc6-4a6b-8161-6d985c064252 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:55.772215 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e7dc2-4fc6-4a6b-8161-6d985c064252 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:55.778956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df3175-3376-4e19-a4ac-bfb1aacfb853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:55.779897 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df3175-3376-4e19-a4ac-bfb1aacfb853 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:55.780183 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df3175-3376-4e19-a4ac-bfb1aacfb853 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:55.780461 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df3175-3376-4e19-a4ac-bfb1aacfb853 admin admin] PATH_INFO: `/v3/projects/c241599785234c62af723fe561c669c6/users/f417d7c9ad294948b2774846f9a1d239/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:55.782278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-b44961ca-bb06-4811-a446-a2835da2c92c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:55.782560 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df3175-3376-4e19-a4ac-bfb1aacfb853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c241599785234c62af723fe561c669c6, user_id=f417d7c9ad294948b2774846f9a1d239, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:55.783966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b44961ca-bb06-4811-a446-a2835da2c92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:55.785311 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df3175-3376-4e19-a4ac-bfb1aacfb853 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:55.801538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1104/2198] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:55.806796 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1094/2198] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:55 2022] PUT /identity/v3/projects/c241599785234c62af723fe561c669c6/users/f417d7c9ad294948b2774846f9a1d239/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:55.813581 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c4ea7-0d19-4ced-a684-a0a63fc8389b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:55.814179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c4ea7-0d19-4ced-a684-a0a63fc8389b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:55.814666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c4ea7-0d19-4ced-a684-a0a63fc8389b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:55.823083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44961ca-bb06-4811-a446-a2835da2c92c None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:55.847693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44961ca-bb06-4811-a446-a2835da2c92c None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:55.860958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-631c4ea7-0d19-4ced-a684-a0a63fc8389b None None] MFA Rules not processed for user `f417d7c9ad294948b2774846f9a1d239`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:55.863067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-631c4ea7-0d19-4ced-a684-a0a63fc8389b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:55.973150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1095/2199] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:55 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:55.979983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790526c9-7f1b-4ff1-a66b-3b98f3cba906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:55.980807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790526c9-7f1b-4ff1-a66b-3b98f3cba906 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:55.981199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790526c9-7f1b-4ff1-a66b-3b98f3cba906 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:55.981579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790526c9-7f1b-4ff1-a66b-3b98f3cba906 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:55.983037 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790526c9-7f1b-4ff1-a66b-3b98f3cba906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:55.983475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790526c9-7f1b-4ff1-a66b-3b98f3cba906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:55.985339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790526c9-7f1b-4ff1-a66b-3b98f3cba906 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:55.991987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c4ea7-0d19-4ced-a684-a0a63fc8389b None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:55.993407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-631c4ea7-0d19-4ced-a684-a0a63fc8389b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:56.026619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1105/2200] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:56.028372 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790526c9-7f1b-4ff1-a66b-3b98f3cba906 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:56.063119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790526c9-7f1b-4ff1-a66b-3b98f3cba906 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:56.075968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c4ea7-0d19-4ced-a684-a0a63fc8389b None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:56.079603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd432b4-957c-41ed-ac75-09418ced584e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:56.080045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ff5a9-2e31-47c9-b8dc-b0dc54ec7757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:56.080632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd432b4-957c-41ed-ac75-09418ced584e admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:56.080754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ff5a9-2e31-47c9-b8dc-b0dc54ec7757 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:56.081048 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ff5a9-2e31-47c9-b8dc-b0dc54ec7757 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:56.081201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd432b4-957c-41ed-ac75-09418ced584e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:56.081299 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ff5a9-2e31-47c9-b8dc-b0dc54ec7757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:56.081398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd432b4-957c-41ed-ac75-09418ced584e admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:56.082209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd432b4-957c-41ed-ac75-09418ced584e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:56.086939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd432b4-957c-41ed-ac75-09418ced584e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:56.105598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ff5a9-2e31-47c9-b8dc-b0dc54ec7757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:56.107862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ff5a9-2e31-47c9-b8dc-b0dc54ec7757 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:56.140017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1096/2201] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:55 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:56.145942 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ff5a9-2e31-47c9-b8dc-b0dc54ec7757 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:56.146377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-841ff5a9-2e31-47c9-b8dc-b0dc54ec7757 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:56.146991 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1106/2202] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:56 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:56.148031 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd432b4-957c-41ed-ac75-09418ced584e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:56.198624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ff5a9-2e31-47c9-b8dc-b0dc54ec7757 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:56.275298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd432b4-957c-41ed-ac75-09418ced584e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:56.291471 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694fb35-80fe-4915-856b-37c8fa3956cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:56.292433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694fb35-80fe-4915-856b-37c8fa3956cd admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:56.292964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694fb35-80fe-4915-856b-37c8fa3956cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:56.293378 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694fb35-80fe-4915-856b-37c8fa3956cd admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:56.297464 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694fb35-80fe-4915-856b-37c8fa3956cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:56.303487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694fb35-80fe-4915-856b-37c8fa3956cd admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:56.324135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:56.327385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:56.327871 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:56.328375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] PATH_INFO: `/v3/projects/4e3dea64bae54cc1bae11a22214c60fa` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:56.329890 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e3dea64bae54cc1bae11a22214c60fa)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:56.332232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:56.370404 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1107/2203] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:56 2022] POST /identity/v3/users => generated 287 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:56.374251 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694fb35-80fe-4915-856b-37c8fa3956cd admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:56.389736 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b694fb35-80fe-4915-856b-37c8fa3956cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:56.427286 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e3dea64bae54cc1bae11a22214c60fa', 'request_id': 'req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:56.443380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e3dea64bae54cc1bae11a22214c60fa', 'request_id': 'req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:56.464394 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e3dea64bae54cc1bae11a22214c60fa', 'request_id': 'req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:56.482872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] The token cache is being invalidate because project 4e3dea64bae54cc1bae11a22214c60fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:56.483275 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:56.484828 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:56.486551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1097/2204] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:56 2022] DELETE /identity/v3/projects/4e3dea64bae54cc1bae11a22214c60fa => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:56.487397 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:56.490994 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:56.525131 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694fb35-80fe-4915-856b-37c8fa3956cd admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:56.587070 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a789c33-2a68-4f59-b03a-f6273a49321a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:56.587879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a789c33-2a68-4f59-b03a-f6273a49321a service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:56.588267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a789c33-2a68-4f59-b03a-f6273a49321a service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:56.588647 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a789c33-2a68-4f59-b03a-f6273a49321a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:56.590655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9a789c33-2a68-4f59-b03a-f6273a49321a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:56.598414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2ac2f-6f89-423d-b8fd-c8b3d179331b admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:56.621838 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02543e27-20af-4604-a13a-181fccc7a893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:56.623043 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02543e27-20af-4604-a13a-181fccc7a893 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:56.623361 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02543e27-20af-4604-a13a-181fccc7a893 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:56.623645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02543e27-20af-4604-a13a-181fccc7a893 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:56.624463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02543e27-20af-4604-a13a-181fccc7a893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:56.629813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02543e27-20af-4604-a13a-181fccc7a893 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:56.666316 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1098/2205] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:56.667521 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02543e27-20af-4604-a13a-181fccc7a893 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:56.674485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a789c33-2a68-4f59-b03a-f6273a49321a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:56.676558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a789c33-2a68-4f59-b03a-f6273a49321a service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:56.677956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-02543e27-20af-4604-a13a-181fccc7a893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:56.709140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1108/2206] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:56 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:56.709733 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a789c33-2a68-4f59-b03a-f6273a49321a service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:56.715160 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02543e27-20af-4604-a13a-181fccc7a893 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:56.727438 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6efc8b-c3ee-4945-acf9-6443e8066182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:56.728329 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6efc8b-c3ee-4945-acf9-6443e8066182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:56.728653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6efc8b-c3ee-4945-acf9-6443e8066182 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:56.728885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6efc8b-c3ee-4945-acf9-6443e8066182 admin admin] PATH_INFO: `/v3/users/9bfc987f64964d9e9cbdd6be35ef7a5b` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:56.730526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a789c33-2a68-4f59-b03a-f6273a49321a service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:56.732776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6efc8b-c3ee-4945-acf9-6443e8066182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bfc987f64964d9e9cbdd6be35ef7a5b)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:56.733973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6efc8b-c3ee-4945-acf9-6443e8066182 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:56.744127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741a5ad-658a-421e-af09-2af16dab0b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:56.745217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741a5ad-658a-421e-af09-2af16dab0b9e admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:56.745607 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741a5ad-658a-421e-af09-2af16dab0b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:56.746152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741a5ad-658a-421e-af09-2af16dab0b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:56.750218 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741a5ad-658a-421e-af09-2af16dab0b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:56.752136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741a5ad-658a-421e-af09-2af16dab0b9e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:56.775872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1109/2207] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:56.777455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741a5ad-658a-421e-af09-2af16dab0b9e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:56.791946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741a5ad-658a-421e-af09-2af16dab0b9e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:56.800580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a4362-8f8c-45e8-9544-9d0833d6bd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:56.803043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a4362-8f8c-45e8-9544-9d0833d6bd96 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:56.803931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a4362-8f8c-45e8-9544-9d0833d6bd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:56.805714 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a4362-8f8c-45e8-9544-9d0833d6bd96 admin admin] PATH_INFO: `/v3/projects/ff6bb908f8e1445dbc687bad944277f1/users/409772b8ed9c45cea7c5a25cbb8f7a55/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:56.836728 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5c6efc8b-c3ee-4945-acf9-6443e8066182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bfc987f64964d9e9cbdd6be35ef7a5b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:56.847215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a4362-8f8c-45e8-9544-9d0833d6bd96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff6bb908f8e1445dbc687bad944277f1, user_id=409772b8ed9c45cea7c5a25cbb8f7a55, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:56.851226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a4362-8f8c-45e8-9544-9d0833d6bd96 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:56.854445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5c6efc8b-c3ee-4945-acf9-6443e8066182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bfc987f64964d9e9cbdd6be35ef7a5b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:56.855715 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5c6efc8b-c3ee-4945-acf9-6443e8066182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bfc987f64964d9e9cbdd6be35ef7a5b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:56.856868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5c6efc8b-c3ee-4945-acf9-6443e8066182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bfc987f64964d9e9cbdd6be35ef7a5b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:56.871820 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5c6efc8b-c3ee-4945-acf9-6443e8066182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bfc987f64964d9e9cbdd6be35ef7a5b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:56.894843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1110/2208] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:56 2022] PUT /identity/v3/projects/ff6bb908f8e1445dbc687bad944277f1/users/409772b8ed9c45cea7c5a25cbb8f7a55/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:56.904667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ddb43-9047-466e-a381-3b620546e7ec None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:56.904919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ddb43-9047-466e-a381-3b620546e7ec None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:56.905171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ddb43-9047-466e-a381-3b620546e7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:56.906149 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1099/2209] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:56 2022] DELETE /identity/v3/users/9bfc987f64964d9e9cbdd6be35ef7a5b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:56.955174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-9f5ddb43-9047-466e-a381-3b620546e7ec None None] MFA Rules not processed for user `409772b8ed9c45cea7c5a25cbb8f7a55`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:56.956667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9f5ddb43-9047-466e-a381-3b620546e7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:57.085545 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1111/2210] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:57.089600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6efc8b-c3ee-4945-acf9-6443e8066182 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:57.091220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5c6efc8b-c3ee-4945-acf9-6443e8066182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:57.196032 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6efc8b-c3ee-4945-acf9-6443e8066182 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:57.201923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3dc4b1-d36e-4af4-b025-0ca1f7030771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:57.202566 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3dc4b1-d36e-4af4-b025-0ca1f7030771 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:57.202883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3dc4b1-d36e-4af4-b025-0ca1f7030771 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:57.203155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3dc4b1-d36e-4af4-b025-0ca1f7030771 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:57.203894 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3dc4b1-d36e-4af4-b025-0ca1f7030771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:57.205822 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3dc4b1-d36e-4af4-b025-0ca1f7030771 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:57.254349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1100/2211] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:57 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:52:57.255886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ddb43-9047-466e-a381-3b620546e7ec None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:57.267570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ddb43-9047-466e-a381-3b620546e7ec None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:57.326362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aab230-af8e-4529-b646-05f1251858ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:57.327163 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab230-af8e-4529-b646-05f1251858ad admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:57.327517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab230-af8e-4529-b646-05f1251858ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:57.327864 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab230-af8e-4529-b646-05f1251858ad admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:57.328663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aab230-af8e-4529-b646-05f1251858ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:57.330675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aab230-af8e-4529-b646-05f1251858ad admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:57.355463 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1112/2212] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:57 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:57.356437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3dc4b1-d36e-4af4-b025-0ca1f7030771 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:57.367887 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3dc4b1-d36e-4af4-b025-0ca1f7030771 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:57.373458 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0080f9-72d7-4458-bba0-394ef6a0d925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:57.374244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0080f9-72d7-4458-bba0-394ef6a0d925 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:57.375112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0080f9-72d7-4458-bba0-394ef6a0d925 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:57.375112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0080f9-72d7-4458-bba0-394ef6a0d925 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:57.375893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0080f9-72d7-4458-bba0-394ef6a0d925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:57.377585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0080f9-72d7-4458-bba0-394ef6a0d925 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:57.394556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aab230-af8e-4529-b646-05f1251858ad admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:57.395716 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1101/2213] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:52:57.405534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aab230-af8e-4529-b646-05f1251858ad admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:57.411134 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b2475-a25f-4c20-bb8d-83acbbf82363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:57.412318 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b2475-a25f-4c20-bb8d-83acbbf82363 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:57.412613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b2475-a25f-4c20-bb8d-83acbbf82363 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:57.412881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b2475-a25f-4c20-bb8d-83acbbf82363 admin admin] PATH_INFO: `/v3/projects/775698af1ac0460194a1425eb62c8371/users/e3baf2aef94a42f8bfaf585cc9c35c37/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:57.432838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b2475-a25f-4c20-bb8d-83acbbf82363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=775698af1ac0460194a1425eb62c8371, user_id=e3baf2aef94a42f8bfaf585cc9c35c37, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:57.435918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b2475-a25f-4c20-bb8d-83acbbf82363 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:57.454965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1113/2214] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:57 2022] PUT /identity/v3/projects/775698af1ac0460194a1425eb62c8371/users/e3baf2aef94a42f8bfaf585cc9c35c37/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:52:57.458570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0080f9-72d7-4458-bba0-394ef6a0d925 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:57.471521 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0080f9-72d7-4458-bba0-394ef6a0d925 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:57.598915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be1423-7bac-4ee8-9f33-2d17b99ef719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:57.599906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be1423-7bac-4ee8-9f33-2d17b99ef719 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:57.601041 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be1423-7bac-4ee8-9f33-2d17b99ef719 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:57.601429 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be1423-7bac-4ee8-9f33-2d17b99ef719 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:57.602321 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be1423-7bac-4ee8-9f33-2d17b99ef719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:57.604938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be1423-7bac-4ee8-9f33-2d17b99ef719 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:57.623884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b2475-a25f-4c20-bb8d-83acbbf82363 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:57.630985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1102/2215] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:57.637069 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b2475-a25f-4c20-bb8d-83acbbf82363 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:57.648612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723df2e8-9e50-491b-b582-965d35e2c80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:57.649644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be1423-7bac-4ee8-9f33-2d17b99ef719 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:57.649763 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723df2e8-9e50-491b-b582-965d35e2c80a admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:57.650081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723df2e8-9e50-491b-b582-965d35e2c80a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:57.654448 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723df2e8-9e50-491b-b582-965d35e2c80a admin admin] PATH_INFO: `/v3/projects/775698af1ac0460194a1425eb62c8371/users/e3baf2aef94a42f8bfaf585cc9c35c37/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:57.655046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-19be1423-7bac-4ee8-9f33-2d17b99ef719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:57.657025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723df2e8-9e50-491b-b582-965d35e2c80a admin admin] RBAC: Authorizing `identity:create_grant(project_id=775698af1ac0460194a1425eb62c8371, user_id=e3baf2aef94a42f8bfaf585cc9c35c37, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:57.659878 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723df2e8-9e50-491b-b582-965d35e2c80a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:57.686973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be1423-7bac-4ee8-9f33-2d17b99ef719 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:57.691303 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1114/2216] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:57 2022] PUT /identity/v3/projects/775698af1ac0460194a1425eb62c8371/users/e3baf2aef94a42f8bfaf585cc9c35c37/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:57.702134 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214c5d1-8b98-4671-8d41-618bf04ded32 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:57.702500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214c5d1-8b98-4671-8d41-618bf04ded32 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:57.702897 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214c5d1-8b98-4671-8d41-618bf04ded32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:57.776496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-0214c5d1-8b98-4671-8d41-618bf04ded32 None None] MFA Rules not processed for user `e3baf2aef94a42f8bfaf585cc9c35c37`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:57.777075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-0214c5d1-8b98-4671-8d41-618bf04ded32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:57.816572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:57.818350 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:57.818652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:57.819058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] PATH_INFO: `/v3/projects/abcb82a62c34445980fce3b38fa81c6a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:57.820746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] RBAC: Authorizing `identity:delete_project(project_id=abcb82a62c34445980fce3b38fa81c6a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:57.823090 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:57.909496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abcb82a62c34445980fce3b38fa81c6a', 'request_id': 'req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:57.925810 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abcb82a62c34445980fce3b38fa81c6a', 'request_id': 'req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:57.937359 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abcb82a62c34445980fce3b38fa81c6a', 'request_id': 'req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:57.965819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1115/2217] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:57.973891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] The token cache is being invalidate because project abcb82a62c34445980fce3b38fa81c6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:52:57.975746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:57.977564 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:57.983548 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1103/2218] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:52:57 2022] DELETE /identity/v3/projects/abcb82a62c34445980fce3b38fa81c6a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:57.991548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbf770-fbdc-4807-9487-7987fdbb05fd None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:57.991548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbf770-fbdc-4807-9487-7987fdbb05fd None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:57.991548 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbf770-fbdc-4807-9487-7987fdbb05fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:58.043921 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-47cbf770-fbdc-4807-9487-7987fdbb05fd None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:58.044710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:58.046203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:58.065001 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-47cbf770-fbdc-4807-9487-7987fdbb05fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:58.194202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6abe85-0dbc-410f-a2fb-de72d54e5eff admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:58.203885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5c9dc-f5ff-48ef-b2bb-2d7b0ea0ddd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:58.204495 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5c9dc-f5ff-48ef-b2bb-2d7b0ea0ddd1 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:58.204777 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5c9dc-f5ff-48ef-b2bb-2d7b0ea0ddd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:58.205054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5c9dc-f5ff-48ef-b2bb-2d7b0ea0ddd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:58.211195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1116/2219] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:52:58.212410 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-edc5c9dc-f5ff-48ef-b2bb-2d7b0ea0ddd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:58.227848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac72ff-3ac0-4189-9d12-c0b690e5aedc None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:58.229515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac72ff-3ac0-4189-9d12-c0b690e5aedc None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:58.230126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac72ff-3ac0-4189-9d12-c0b690e5aedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:58.263860 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-10ac72ff-3ac0-4189-9d12-c0b690e5aedc None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:58.266252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-10ac72ff-3ac0-4189-9d12-c0b690e5aedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:58.295197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1117/2220] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:58.298670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac72ff-3ac0-4189-9d12-c0b690e5aedc None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:58.332448 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac72ff-3ac0-4189-9d12-c0b690e5aedc None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:58.338569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5c9dc-f5ff-48ef-b2bb-2d7b0ea0ddd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:58.340638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5c9dc-f5ff-48ef-b2bb-2d7b0ea0ddd1 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:58.351834 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55abc78-1ce9-4173-a92b-27679233a427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:58.353701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55abc78-1ce9-4173-a92b-27679233a427 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:58.355280 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55abc78-1ce9-4173-a92b-27679233a427 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:58.355919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55abc78-1ce9-4173-a92b-27679233a427 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:58.360845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55abc78-1ce9-4173-a92b-27679233a427 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:52:58.361169 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55abc78-1ce9-4173-a92b-27679233a427 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:58.367918 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1104/2221] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:58.371378 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55abc78-1ce9-4173-a92b-27679233a427 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:58.410843 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1118/2222] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:52:58.415644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5c9dc-f5ff-48ef-b2bb-2d7b0ea0ddd1 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:58.439087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5c9dc-f5ff-48ef-b2bb-2d7b0ea0ddd1 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:58.462326 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8db771-3348-44eb-ab47-6e9ab00f9afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:58.491183 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8db771-3348-44eb-ab47-6e9ab00f9afa admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:58.492686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8db771-3348-44eb-ab47-6e9ab00f9afa admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:58.493041 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8db771-3348-44eb-ab47-6e9ab00f9afa admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:58.493767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8db771-3348-44eb-ab47-6e9ab00f9afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:58.496509 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8db771-3348-44eb-ab47-6e9ab00f9afa admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:58.573763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1105/2223] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:58 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:58.574938 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55abc78-1ce9-4173-a92b-27679233a427 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:58.588485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55abc78-1ce9-4173-a92b-27679233a427 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:58.723690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14441e-831a-400a-84fa-eaf22a063683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:58.723690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14441e-831a-400a-84fa-eaf22a063683 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:58.723690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14441e-831a-400a-84fa-eaf22a063683 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:58.723690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14441e-831a-400a-84fa-eaf22a063683 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:58.724571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14441e-831a-400a-84fa-eaf22a063683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:58.724571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14441e-831a-400a-84fa-eaf22a063683 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:58.756235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1119/2224] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:58 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:58.758567 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8db771-3348-44eb-ab47-6e9ab00f9afa admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:58.775214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8db771-3348-44eb-ab47-6e9ab00f9afa admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:58.781141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba4a35-9119-4b33-bbcf-2683aaca6289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:58.782098 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba4a35-9119-4b33-bbcf-2683aaca6289 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:58.782653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba4a35-9119-4b33-bbcf-2683aaca6289 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:58.783121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba4a35-9119-4b33-bbcf-2683aaca6289 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:58.784018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba4a35-9119-4b33-bbcf-2683aaca6289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:58.786100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba4a35-9119-4b33-bbcf-2683aaca6289 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:58.805998 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1106/2225] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:58.807565 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14441e-831a-400a-84fa-eaf22a063683 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:58.819758 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14441e-831a-400a-84fa-eaf22a063683 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:58.826493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d019fd-71e5-4389-b6a4-dbac15caa19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:58.827423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d019fd-71e5-4389-b6a4-dbac15caa19c admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:58.828113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d019fd-71e5-4389-b6a4-dbac15caa19c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:58.828460 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d019fd-71e5-4389-b6a4-dbac15caa19c admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:58.829183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d019fd-71e5-4389-b6a4-dbac15caa19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:58.832348 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d019fd-71e5-4389-b6a4-dbac15caa19c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:58.867821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba4a35-9119-4b33-bbcf-2683aaca6289 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:58.868573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1120/2226] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:52:58.883585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba4a35-9119-4b33-bbcf-2683aaca6289 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:58.896905 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752149e-f136-4d8c-83df-b951031308f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:58.897907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752149e-f136-4d8c-83df-b951031308f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:58.898064 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752149e-f136-4d8c-83df-b951031308f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:58.898687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752149e-f136-4d8c-83df-b951031308f9 admin admin] PATH_INFO: `/v3/projects/7690caec8c7c43b59b83e94ae25bfc4e/users/8b43b811255a43aa99597c809e1fb453/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:58.935457 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752149e-f136-4d8c-83df-b951031308f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7690caec8c7c43b59b83e94ae25bfc4e, user_id=8b43b811255a43aa99597c809e1fb453, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:58.940004 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752149e-f136-4d8c-83df-b951031308f9 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:58.970291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1107/2227] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:58 2022] PUT /identity/v3/projects/7690caec8c7c43b59b83e94ae25bfc4e/users/8b43b811255a43aa99597c809e1fb453/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:58.980995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fa5e2-56b9-44d4-9c75-a9c374d640b5 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:58.981284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fa5e2-56b9-44d4-9c75-a9c374d640b5 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:58.981629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fa5e2-56b9-44d4-9c75-a9c374d640b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:59.016090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-466fa5e2-56b9-44d4-9c75-a9c374d640b5 None None] MFA Rules not processed for user `8b43b811255a43aa99597c809e1fb453`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:59.017408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-466fa5e2-56b9-44d4-9c75-a9c374d640b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:59.108013 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1121/2228] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:59.112881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752149e-f136-4d8c-83df-b951031308f9 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:59.172075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fa5e2-56b9-44d4-9c75-a9c374d640b5 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:59.173522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-466fa5e2-56b9-44d4-9c75-a9c374d640b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:59.226075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752149e-f136-4d8c-83df-b951031308f9 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:59.231480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea33efd-a52a-4f23-a90a-4368f8c2db6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:59.232826 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea33efd-a52a-4f23-a90a-4368f8c2db6d admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:59.233108 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea33efd-a52a-4f23-a90a-4368f8c2db6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:59.233384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea33efd-a52a-4f23-a90a-4368f8c2db6d admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:59.234089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea33efd-a52a-4f23-a90a-4368f8c2db6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:59.236123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea33efd-a52a-4f23-a90a-4368f8c2db6d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:59.266830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fa5e2-56b9-44d4-9c75-a9c374d640b5 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:59.267049 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1108/2229] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:52:59 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:59.268107 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea33efd-a52a-4f23-a90a-4368f8c2db6d admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:59.332733 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ed5dd-e01c-4278-8c91-ed737d291c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:59.333612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ed5dd-e01c-4278-8c91-ed737d291c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:59.333881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ed5dd-e01c-4278-8c91-ed737d291c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:59.334152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ed5dd-e01c-4278-8c91-ed737d291c51 admin admin] PATH_INFO: `/v3/users/8784e340a42f40fbba60fefa4ef50055` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:59.335744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ed5dd-e01c-4278-8c91-ed737d291c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8784e340a42f40fbba60fefa4ef50055)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:59.337883 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ed5dd-e01c-4278-8c91-ed737d291c51 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:59.338611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea33efd-a52a-4f23-a90a-4368f8c2db6d admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:59.344189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b026ccc-2ae5-4a0a-8cec-4aaccf6ac2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:59.344868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026ccc-2ae5-4a0a-8cec-4aaccf6ac2b7 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:59.345931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026ccc-2ae5-4a0a-8cec-4aaccf6ac2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:59.345931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026ccc-2ae5-4a0a-8cec-4aaccf6ac2b7 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:59.346122 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b026ccc-2ae5-4a0a-8cec-4aaccf6ac2b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:59.348464 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b026ccc-2ae5-4a0a-8cec-4aaccf6ac2b7 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:59.376156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1109/2230] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:52:59 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:52:59.380874 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b026ccc-2ae5-4a0a-8cec-4aaccf6ac2b7 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:59.393306 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b026ccc-2ae5-4a0a-8cec-4aaccf6ac2b7 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:59.398139 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d94df-b4b3-4217-8f8e-e6bb2ad3a76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:59.399077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d94df-b4b3-4217-8f8e-e6bb2ad3a76a admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:59.399424 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d94df-b4b3-4217-8f8e-e6bb2ad3a76a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:59.399701 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d94df-b4b3-4217-8f8e-e6bb2ad3a76a admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:59.400556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d94df-b4b3-4217-8f8e-e6bb2ad3a76a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:59.401419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-845ed5dd-e01c-4278-8c91-ed737d291c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8784e340a42f40fbba60fefa4ef50055'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:59.408682 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d94df-b4b3-4217-8f8e-e6bb2ad3a76a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:59.420094 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-845ed5dd-e01c-4278-8c91-ed737d291c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8784e340a42f40fbba60fefa4ef50055'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:59.421016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-845ed5dd-e01c-4278-8c91-ed737d291c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8784e340a42f40fbba60fefa4ef50055'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:59.421754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1110/2231] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:59.421866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-845ed5dd-e01c-4278-8c91-ed737d291c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8784e340a42f40fbba60fefa4ef50055'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:59.426611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d94df-b4b3-4217-8f8e-e6bb2ad3a76a admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:59.428851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8b1d94df-b4b3-4217-8f8e-e6bb2ad3a76a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:59.434452 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-845ed5dd-e01c-4278-8c91-ed737d291c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8784e340a42f40fbba60fefa4ef50055'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:52:59.446676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d94df-b4b3-4217-8f8e-e6bb2ad3a76a admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:59.453663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1122/2232] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:52:59 2022] DELETE /identity/v3/users/8784e340a42f40fbba60fefa4ef50055 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:59.456649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab692b96-a502-490d-ab4b-47b2a8012250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:59.458915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfb6f9-f2e5-413e-a825-aa2b21c96e23 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:59.459327 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfb6f9-f2e5-413e-a825-aa2b21c96e23 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:59.459720 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfb6f9-f2e5-413e-a825-aa2b21c96e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:59.460709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab692b96-a502-490d-ab4b-47b2a8012250 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:59.461187 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab692b96-a502-490d-ab4b-47b2a8012250 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:59.461688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab692b96-a502-490d-ab4b-47b2a8012250 admin admin] PATH_INFO: `/v3/projects/1688f45086ca4cfdaf54863bc16ce590/users/9363165e1cd54b47a11fb59b0c8b6b67/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:59.481217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-ebcfb6f9-f2e5-413e-a825-aa2b21c96e23 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:59.482562 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ebcfb6f9-f2e5-413e-a825-aa2b21c96e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:59.496585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab692b96-a502-490d-ab4b-47b2a8012250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1688f45086ca4cfdaf54863bc16ce590, user_id=9363165e1cd54b47a11fb59b0c8b6b67, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:59.499645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab692b96-a502-490d-ab4b-47b2a8012250 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:59.529440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1111/2233] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:59 2022] PUT /identity/v3/projects/1688f45086ca4cfdaf54863bc16ce590/users/9363165e1cd54b47a11fb59b0c8b6b67/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:59.536483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab692b96-a502-490d-ab4b-47b2a8012250 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:59.604405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1123/2234] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:52:59.614956 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfb6f9-f2e5-413e-a825-aa2b21c96e23 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:59.618017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ebcfb6f9-f2e5-413e-a825-aa2b21c96e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:59.671708 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab692b96-a502-490d-ab4b-47b2a8012250 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:59.688697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bd49b-2611-4ebd-9395-770070b2114e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:59.691250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bd49b-2611-4ebd-9395-770070b2114e admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:59.692184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bd49b-2611-4ebd-9395-770070b2114e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:59.692497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bd49b-2611-4ebd-9395-770070b2114e admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:59.693267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bd49b-2611-4ebd-9395-770070b2114e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:59.696166 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bd49b-2611-4ebd-9395-770070b2114e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:59.717541 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfb6f9-f2e5-413e-a825-aa2b21c96e23 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:59.731234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1112/2235] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:52:59.734381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bd49b-2611-4ebd-9395-770070b2114e admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:52:59.776086 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bd49b-2611-4ebd-9395-770070b2114e admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:52:59.790621 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76c791-845f-4204-9cb3-e6534f12c62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:59.791563 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c791-845f-4204-9cb3-e6534f12c62e admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:59.791855 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c791-845f-4204-9cb3-e6534f12c62e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:59.792240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c791-845f-4204-9cb3-e6534f12c62e admin admin] PATH_INFO: `/v3/projects/1688f45086ca4cfdaf54863bc16ce590/users/9363165e1cd54b47a11fb59b0c8b6b67/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:59.795075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76c791-845f-4204-9cb3-e6534f12c62e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1688f45086ca4cfdaf54863bc16ce590, user_id=9363165e1cd54b47a11fb59b0c8b6b67, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:59.801468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76c791-845f-4204-9cb3-e6534f12c62e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:59.830029 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1113/2236] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:52:59 2022] PUT /identity/v3/projects/1688f45086ca4cfdaf54863bc16ce590/users/9363165e1cd54b47a11fb59b0c8b6b67/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:52:59.842395 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702ebaa-bf54-40f2-ba19-0a86f1d652e1 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:59.842909 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702ebaa-bf54-40f2-ba19-0a86f1d652e1 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:59.843367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702ebaa-bf54-40f2-ba19-0a86f1d652e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:59.848243 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fdfb59-fabe-4533-b0cf-54916cc4e0d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:52:59.849028 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdfb59-fabe-4533-b0cf-54916cc4e0d3 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:59.849413 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdfb59-fabe-4533-b0cf-54916cc4e0d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:59.849792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdfb59-fabe-4533-b0cf-54916cc4e0d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:59.882256 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fdfb59-fabe-4533-b0cf-54916cc4e0d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:52:59.883956 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fdfb59-fabe-4533-b0cf-54916cc4e0d3 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:52:59.899265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1124/2237] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:52:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:52:59.901450 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c84827-ce74-46cf-8efe-72413c982b1e None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:52:59.901890 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c84827-ce74-46cf-8efe-72413c982b1e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:52:59.902401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c84827-ce74-46cf-8efe-72413c982b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:52:59.927790 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-b702ebaa-bf54-40f2-ba19-0a86f1d652e1 None None] MFA Rules not processed for user `9363165e1cd54b47a11fb59b0c8b6b67`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:52:59.929246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b702ebaa-bf54-40f2-ba19-0a86f1d652e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:52:59.995351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-98c84827-ce74-46cf-8efe-72413c982b1e None None] MFA Rules not processed for user `42a643613f144989b42be5865e82e4f2`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:00.001652 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-98c84827-ce74-46cf-8efe-72413c982b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:00.126136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1114/2238] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:53:00.140025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702ebaa-bf54-40f2-ba19-0a86f1d652e1 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:00.141442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b702ebaa-bf54-40f2-ba19-0a86f1d652e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:00.154197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1125/2239] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:52:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:00.250652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702ebaa-bf54-40f2-ba19-0a86f1d652e1 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:00.254835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a562550-dcb2-4016-865c-cf244a834f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:00.255488 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a562550-dcb2-4016-865c-cf244a834f3e service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:00.255774 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a562550-dcb2-4016-865c-cf244a834f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:00.256062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a562550-dcb2-4016-865c-cf244a834f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:00.257932 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4a562550-dcb2-4016-865c-cf244a834f3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:00.335077 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c84827-ce74-46cf-8efe-72413c982b1e None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:00.336659 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-98c84827-ce74-46cf-8efe-72413c982b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:00.380178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a562550-dcb2-4016-865c-cf244a834f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:00.381868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a562550-dcb2-4016-865c-cf244a834f3e service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:00.401204 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1115/2240] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:53:00.454064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c84827-ce74-46cf-8efe-72413c982b1e None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:00.463152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd21718-04ef-42c6-8e6f-bb63cd9152ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:00.463957 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd21718-04ef-42c6-8e6f-bb63cd9152ff service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:00.464245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd21718-04ef-42c6-8e6f-bb63cd9152ff service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:00.464522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd21718-04ef-42c6-8e6f-bb63cd9152ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:00.467197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-dbd21718-04ef-42c6-8e6f-bb63cd9152ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:00.571767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a562550-dcb2-4016-865c-cf244a834f3e service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:00.573207 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4a562550-dcb2-4016-865c-cf244a834f3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:00.616580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd21718-04ef-42c6-8e6f-bb63cd9152ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:00.618740 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd21718-04ef-42c6-8e6f-bb63cd9152ff service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:00.635891 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1126/2241] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:53:00.707026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a562550-dcb2-4016-865c-cf244a834f3e service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:00.717702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:00.718821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:00.718948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:00.719813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] PATH_INFO: `/v3/projects/0e4da8602d7140f5a3feb7713b96aef8` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:00.722120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e4da8602d7140f5a3feb7713b96aef8)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:00.725432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:00.832595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e4da8602d7140f5a3feb7713b96aef8', 'request_id': 'req-529c1aef-f440-48dc-a431-a765478153c2'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:00.861076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e4da8602d7140f5a3feb7713b96aef8', 'request_id': 'req-529c1aef-f440-48dc-a431-a765478153c2'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:00.874685 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e4da8602d7140f5a3feb7713b96aef8', 'request_id': 'req-529c1aef-f440-48dc-a431-a765478153c2'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:00.893805 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] The token cache is being invalidate because project 0e4da8602d7140f5a3feb7713b96aef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:00.895252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:00.897759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:00.901947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1116/2242] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:00 2022] DELETE /identity/v3/projects/0e4da8602d7140f5a3feb7713b96aef8 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:53:00.906961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd21718-04ef-42c6-8e6f-bb63cd9152ff service nova] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:00.926930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd21718-04ef-42c6-8e6f-bb63cd9152ff service nova] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:01.019198 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c19729-4517-470f-a88f-bea4e3cf3edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:01.020110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c19729-4517-470f-a88f-bea4e3cf3edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:01.020389 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c19729-4517-470f-a88f-bea4e3cf3edd admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:01.020776 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c19729-4517-470f-a88f-bea4e3cf3edd admin admin] PATH_INFO: `/v3/users/1bde078c591e47a39e7c360718667af1` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:01.022384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c19729-4517-470f-a88f-bea4e3cf3edd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bde078c591e47a39e7c360718667af1)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:01.024650 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c19729-4517-470f-a88f-bea4e3cf3edd admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:01.084986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-89c19729-4517-470f-a88f-bea4e3cf3edd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bde078c591e47a39e7c360718667af1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:01.102697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-89c19729-4517-470f-a88f-bea4e3cf3edd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bde078c591e47a39e7c360718667af1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:01.103416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-89c19729-4517-470f-a88f-bea4e3cf3edd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bde078c591e47a39e7c360718667af1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:01.104018 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-89c19729-4517-470f-a88f-bea4e3cf3edd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bde078c591e47a39e7c360718667af1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:01.113645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-89c19729-4517-470f-a88f-bea4e3cf3edd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bde078c591e47a39e7c360718667af1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:01.139684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1127/2243] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:00 2022] DELETE /identity/v3/users/1bde078c591e47a39e7c360718667af1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:01.469174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:01.471044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:01.493843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c1aef-f440-48dc-a431-a765478153c2 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:01.625760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf1a88-25a7-4360-a1b9-a0fcb081937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:01.627220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf1a88-25a7-4360-a1b9-a0fcb081937c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:01.627431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf1a88-25a7-4360-a1b9-a0fcb081937c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:01.627772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf1a88-25a7-4360-a1b9-a0fcb081937c admin admin] PATH_INFO: `/v3/users/409772b8ed9c45cea7c5a25cbb8f7a55` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:01.629217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf1a88-25a7-4360-a1b9-a0fcb081937c admin admin] RBAC: Authorizing `identity:delete_user(user_id=409772b8ed9c45cea7c5a25cbb8f7a55)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:01.631425 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf1a88-25a7-4360-a1b9-a0fcb081937c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:01.732276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ebaf1a88-25a7-4360-a1b9-a0fcb081937c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '409772b8ed9c45cea7c5a25cbb8f7a55'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:01.759161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ebaf1a88-25a7-4360-a1b9-a0fcb081937c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '409772b8ed9c45cea7c5a25cbb8f7a55'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:01.759990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ebaf1a88-25a7-4360-a1b9-a0fcb081937c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '409772b8ed9c45cea7c5a25cbb8f7a55'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:01.761197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ebaf1a88-25a7-4360-a1b9-a0fcb081937c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '409772b8ed9c45cea7c5a25cbb8f7a55'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:01.790490 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ebaf1a88-25a7-4360-a1b9-a0fcb081937c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '409772b8ed9c45cea7c5a25cbb8f7a55'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:01.819868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1117/2244] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:01 2022] DELETE /identity/v3/users/409772b8ed9c45cea7c5a25cbb8f7a55 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:01.825003 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee4ff3-0fa5-427c-8d37-009af23f0369 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:01.826013 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee4ff3-0fa5-427c-8d37-009af23f0369 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:01.826395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee4ff3-0fa5-427c-8d37-009af23f0369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:01.833230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf1a88-25a7-4360-a1b9-a0fcb081937c admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:01.835285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ebaf1a88-25a7-4360-a1b9-a0fcb081937c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:01.861086 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-3aee4ff3-0fa5-427c-8d37-009af23f0369 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:01.864193 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3aee4ff3-0fa5-427c-8d37-009af23f0369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:01.966579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1128/2245] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:01.976498 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf1a88-25a7-4360-a1b9-a0fcb081937c admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:01.978120 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee4ff3-0fa5-427c-8d37-009af23f0369 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:01.981836 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3aee4ff3-0fa5-427c-8d37-009af23f0369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:01.987748 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:01.988780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:01.990404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:01.990693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] PATH_INFO: `/v3/projects/33951e60f0cc4a2bb3efc3fdb9d03193` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:01.992026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=33951e60f0cc4a2bb3efc3fdb9d03193)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:01.993962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:02.089078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33951e60f0cc4a2bb3efc3fdb9d03193', 'request_id': 'req-a6205817-023f-4e08-8742-0c93fdabaa3f'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:02.102832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33951e60f0cc4a2bb3efc3fdb9d03193', 'request_id': 'req-a6205817-023f-4e08-8742-0c93fdabaa3f'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:02.118653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee4ff3-0fa5-427c-8d37-009af23f0369 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:02.120604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33951e60f0cc4a2bb3efc3fdb9d03193', 'request_id': 'req-a6205817-023f-4e08-8742-0c93fdabaa3f'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:02.157512 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] The token cache is being invalidate because project 33951e60f0cc4a2bb3efc3fdb9d03193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:02.157836 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:02.158647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:02.163486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1118/2246] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:01 2022] DELETE /identity/v3/projects/33951e60f0cc4a2bb3efc3fdb9d03193 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:02.164594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:02.166162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:02.282101 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651a8bc-519e-4a62-8f91-18b3c9e49429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:02.282626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651a8bc-519e-4a62-8f91-18b3c9e49429 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:02.282970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651a8bc-519e-4a62-8f91-18b3c9e49429 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:02.283307 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651a8bc-519e-4a62-8f91-18b3c9e49429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:02.290051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b651a8bc-519e-4a62-8f91-18b3c9e49429 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:02.390405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6205817-023f-4e08-8742-0c93fdabaa3f admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:02.403843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af4321-ba8c-4f7b-b140-3c38855bfd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:02.406197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af4321-ba8c-4f7b-b140-3c38855bfd1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:02.407273 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af4321-ba8c-4f7b-b140-3c38855bfd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:02.408522 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af4321-ba8c-4f7b-b140-3c38855bfd1d admin admin] PATH_INFO: `/v3/users/81c403619891468fbe8f4b808388d1f9` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:02.417093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af4321-ba8c-4f7b-b140-3c38855bfd1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=81c403619891468fbe8f4b808388d1f9)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:02.428512 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af4321-ba8c-4f7b-b140-3c38855bfd1d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:02.493507 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651a8bc-519e-4a62-8f91-18b3c9e49429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:02.495138 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651a8bc-519e-4a62-8f91-18b3c9e49429 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:02.510312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1129/2247] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:02.517812 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b6af4321-ba8c-4f7b-b140-3c38855bfd1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81c403619891468fbe8f4b808388d1f9'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:02.538795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b6af4321-ba8c-4f7b-b140-3c38855bfd1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81c403619891468fbe8f4b808388d1f9'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:02.543018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b6af4321-ba8c-4f7b-b140-3c38855bfd1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81c403619891468fbe8f4b808388d1f9'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:02.544619 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b6af4321-ba8c-4f7b-b140-3c38855bfd1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81c403619891468fbe8f4b808388d1f9'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:02.561498 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b6af4321-ba8c-4f7b-b140-3c38855bfd1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81c403619891468fbe8f4b808388d1f9'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:02.581576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1119/2248] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:02 2022] DELETE /identity/v3/users/81c403619891468fbe8f4b808388d1f9 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:02.978854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651a8bc-519e-4a62-8f91-18b3c9e49429 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:02.994735 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651a8bc-519e-4a62-8f91-18b3c9e49429 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:03.142605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af4321-ba8c-4f7b-b140-3c38855bfd1d admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:03.146028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b6af4321-ba8c-4f7b-b140-3c38855bfd1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:03.179532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d679d-d20c-4c15-b96d-bd2a267972f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:03.183014 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d679d-d20c-4c15-b96d-bd2a267972f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:03.183474 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d679d-d20c-4c15-b96d-bd2a267972f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:03.184921 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d679d-d20c-4c15-b96d-bd2a267972f4 admin admin] PATH_INFO: `/v3/projects/8f1fcc06288f48e7b3660f907c8d9ff5` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:03.195680 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d679d-d20c-4c15-b96d-bd2a267972f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f1fcc06288f48e7b3660f907c8d9ff5)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:03.198655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d679d-d20c-4c15-b96d-bd2a267972f4 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:03.304339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-af8d679d-d20c-4c15-b96d-bd2a267972f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f1fcc06288f48e7b3660f907c8d9ff5', 'request_id': 'req-af8d679d-d20c-4c15-b96d-bd2a267972f4'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.312556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-af8d679d-d20c-4c15-b96d-bd2a267972f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f1fcc06288f48e7b3660f907c8d9ff5', 'request_id': 'req-af8d679d-d20c-4c15-b96d-bd2a267972f4'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.327794 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-af8d679d-d20c-4c15-b96d-bd2a267972f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f1fcc06288f48e7b3660f907c8d9ff5', 'request_id': 'req-af8d679d-d20c-4c15-b96d-bd2a267972f4'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.346166 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-af8d679d-d20c-4c15-b96d-bd2a267972f4 admin admin] The token cache is being invalidate because project 8f1fcc06288f48e7b3660f907c8d9ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:03.346467 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-af8d679d-d20c-4c15-b96d-bd2a267972f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.347251 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-af8d679d-d20c-4c15-b96d-bd2a267972f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.348544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af4321-ba8c-4f7b-b140-3c38855bfd1d admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:03.349724 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1130/2249] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:02 2022] DELETE /identity/v3/projects/8f1fcc06288f48e7b3660f907c8d9ff5 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:03.367845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a21fea-51c8-4c74-afba-3d33e9f386cb None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:03.369061 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a21fea-51c8-4c74-afba-3d33e9f386cb None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:03.370192 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a21fea-51c8-4c74-afba-3d33e9f386cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:03.404539 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-15a21fea-51c8-4c74-afba-3d33e9f386cb None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:03.405870 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-15a21fea-51c8-4c74-afba-3d33e9f386cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:03.437001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:03.438732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:03.438732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:03.438732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] PATH_INFO: `/v3/projects/ff6bb908f8e1445dbc687bad944277f1` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:03.439966 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff6bb908f8e1445dbc687bad944277f1)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:03.441981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:03.476542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1131/2250] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:03.485542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e91e2a-5c5c-482f-9ec2-e180081aec3e None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:03.485837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e91e2a-5c5c-482f-9ec2-e180081aec3e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:03.486122 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e91e2a-5c5c-482f-9ec2-e180081aec3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:03.496922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff6bb908f8e1445dbc687bad944277f1', 'request_id': 'req-2aacc685-3616-48f9-b3bf-843631d0982b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.505293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-42e91e2a-5c5c-482f-9ec2-e180081aec3e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:03.505817 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff6bb908f8e1445dbc687bad944277f1', 'request_id': 'req-2aacc685-3616-48f9-b3bf-843631d0982b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.506514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-42e91e2a-5c5c-482f-9ec2-e180081aec3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:03.515281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff6bb908f8e1445dbc687bad944277f1', 'request_id': 'req-2aacc685-3616-48f9-b3bf-843631d0982b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.529703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] The token cache is being invalidate because project ff6bb908f8e1445dbc687bad944277f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:03.530374 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.530666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.535102 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1120/2251] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:03 2022] DELETE /identity/v3/projects/ff6bb908f8e1445dbc687bad944277f1 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:03.536542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:03.537984 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:03.576533 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1132/2252] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:53:03.580788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e91e2a-5c5c-482f-9ec2-e180081aec3e None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:03.589757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e91e2a-5c5c-482f-9ec2-e180081aec3e None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:03.595623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fea743-cf00-454b-9144-ad2497e0e38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:03.596301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fea743-cf00-454b-9144-ad2497e0e38d admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:03.596581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fea743-cf00-454b-9144-ad2497e0e38d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:03.596855 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fea743-cf00-454b-9144-ad2497e0e38d admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:03.597585 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fea743-cf00-454b-9144-ad2497e0e38d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:53:03.597886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fea743-cf00-454b-9144-ad2497e0e38d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:03.599542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fea743-cf00-454b-9144-ad2497e0e38d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:03.604990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacc685-3616-48f9-b3bf-843631d0982b admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:03.610572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb790de-451d-488a-901c-7c7cf0dff928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:03.614768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb790de-451d-488a-901c-7c7cf0dff928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:03.615027 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1133/2253] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:53:03.615405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb790de-451d-488a-901c-7c7cf0dff928 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:03.615405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb790de-451d-488a-901c-7c7cf0dff928 admin admin] PATH_INFO: `/v3/users/e3baf2aef94a42f8bfaf585cc9c35c37` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:03.615405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb790de-451d-488a-901c-7c7cf0dff928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3baf2aef94a42f8bfaf585cc9c35c37)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:03.616016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fea743-cf00-454b-9144-ad2497e0e38d admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:03.616254 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb790de-451d-488a-901c-7c7cf0dff928 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:03.636056 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fea743-cf00-454b-9144-ad2497e0e38d admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:03.642143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a3546-f5e1-4eb2-aede-1e77a72f401f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:03.642857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a3546-f5e1-4eb2-aede-1e77a72f401f admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:03.643185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a3546-f5e1-4eb2-aede-1e77a72f401f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:03.643487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a3546-f5e1-4eb2-aede-1e77a72f401f admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:03.644242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a3546-f5e1-4eb2-aede-1e77a72f401f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:03.646263 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a3546-f5e1-4eb2-aede-1e77a72f401f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:03.672854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2eb790de-451d-488a-901c-7c7cf0dff928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3baf2aef94a42f8bfaf585cc9c35c37'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.678574 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1134/2254] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:53:03 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:53:03.679900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a3546-f5e1-4eb2-aede-1e77a72f401f admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:03.686956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2eb790de-451d-488a-901c-7c7cf0dff928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3baf2aef94a42f8bfaf585cc9c35c37'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.688653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2eb790de-451d-488a-901c-7c7cf0dff928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3baf2aef94a42f8bfaf585cc9c35c37'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.689816 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2eb790de-451d-488a-901c-7c7cf0dff928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3baf2aef94a42f8bfaf585cc9c35c37'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.698210 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-2eb790de-451d-488a-901c-7c7cf0dff928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3baf2aef94a42f8bfaf585cc9c35c37'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:03.720135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1121/2255] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:03 2022] DELETE /identity/v3/users/e3baf2aef94a42f8bfaf585cc9c35c37 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:03.749547 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a3546-f5e1-4eb2-aede-1e77a72f401f admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:03.827138 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab0600-e13a-4286-a60a-b3f7ebb2a503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:03.831904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab0600-e13a-4286-a60a-b3f7ebb2a503 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:03.832204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab0600-e13a-4286-a60a-b3f7ebb2a503 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:03.832628 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab0600-e13a-4286-a60a-b3f7ebb2a503 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:03.833478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab0600-e13a-4286-a60a-b3f7ebb2a503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:03.836082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab0600-e13a-4286-a60a-b3f7ebb2a503 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:03.859267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1135/2256] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:53:03 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:03.863167 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb790de-451d-488a-901c-7c7cf0dff928 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:03.864653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2eb790de-451d-488a-901c-7c7cf0dff928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:03.881585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb790de-451d-488a-901c-7c7cf0dff928 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:03.886827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e08f7-e47f-4c4e-a470-d03b5a54f84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:03.887520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e08f7-e47f-4c4e-a470-d03b5a54f84c admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:03.887772 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e08f7-e47f-4c4e-a470-d03b5a54f84c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:03.888049 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e08f7-e47f-4c4e-a470-d03b5a54f84c admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:03.888776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e08f7-e47f-4c4e-a470-d03b5a54f84c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:03.890429 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e08f7-e47f-4c4e-a470-d03b5a54f84c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:03.903060 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1122/2257] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:53:03.907396 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab0600-e13a-4286-a60a-b3f7ebb2a503 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:03.917793 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab0600-e13a-4286-a60a-b3f7ebb2a503 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:03.923345 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b3114-5f3c-4ee6-b8b0-91d1897091f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:03.924021 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b3114-5f3c-4ee6-b8b0-91d1897091f9 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:03.924377 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b3114-5f3c-4ee6-b8b0-91d1897091f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:03.924724 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b3114-5f3c-4ee6-b8b0-91d1897091f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:03.925669 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b3114-5f3c-4ee6-b8b0-91d1897091f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:03.927408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b3114-5f3c-4ee6-b8b0-91d1897091f9 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:03.949451 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1136/2258] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:03.950970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e08f7-e47f-4c4e-a470-d03b5a54f84c admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:03.963312 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e08f7-e47f-4c4e-a470-d03b5a54f84c admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:03.968489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ad51f-99df-41fb-834c-a02e97856794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:03.969400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ad51f-99df-41fb-834c-a02e97856794 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:03.969676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ad51f-99df-41fb-834c-a02e97856794 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:03.969954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ad51f-99df-41fb-834c-a02e97856794 admin admin] PATH_INFO: `/v3/projects/8fea379bf5c74e72b3b25a35f8327535/users/4d461c00682d4024911af8c38046cd75/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:03.986344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ad51f-99df-41fb-834c-a02e97856794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fea379bf5c74e72b3b25a35f8327535, user_id=4d461c00682d4024911af8c38046cd75, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:03.989094 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ad51f-99df-41fb-834c-a02e97856794 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:04.008083 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1123/2259] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:03 2022] PUT /identity/v3/projects/8fea379bf5c74e72b3b25a35f8327535/users/4d461c00682d4024911af8c38046cd75/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:04.018695 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf375e89-e7ec-4a4d-a8fc-b14976193b8b None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:04.019286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf375e89-e7ec-4a4d-a8fc-b14976193b8b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:04.019642 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf375e89-e7ec-4a4d-a8fc-b14976193b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:04.032272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ad51f-99df-41fb-834c-a02e97856794 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:04.034025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-273ad51f-99df-41fb-834c-a02e97856794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:04.052081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-cf375e89-e7ec-4a4d-a8fc-b14976193b8b None None] MFA Rules not processed for user `4d461c00682d4024911af8c38046cd75`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:04.053298 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-cf375e89-e7ec-4a4d-a8fc-b14976193b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:04.111471 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ad51f-99df-41fb-834c-a02e97856794 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:04.116910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d505806-8d4e-4893-a76c-3a5a7611e6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:04.117782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d505806-8d4e-4893-a76c-3a5a7611e6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:04.118139 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d505806-8d4e-4893-a76c-3a5a7611e6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:04.118336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d505806-8d4e-4893-a76c-3a5a7611e6ce admin admin] PATH_INFO: `/v3/projects/775698af1ac0460194a1425eb62c8371` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:04.119563 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d505806-8d4e-4893-a76c-3a5a7611e6ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=775698af1ac0460194a1425eb62c8371)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:04.121518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d505806-8d4e-4893-a76c-3a5a7611e6ce admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:04.129467 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1137/2260] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:04.140931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf375e89-e7ec-4a4d-a8fc-b14976193b8b None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:04.142262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-cf375e89-e7ec-4a4d-a8fc-b14976193b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:04.176026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-1d505806-8d4e-4893-a76c-3a5a7611e6ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '775698af1ac0460194a1425eb62c8371', 'request_id': 'req-1d505806-8d4e-4893-a76c-3a5a7611e6ce'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:04.182164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-1d505806-8d4e-4893-a76c-3a5a7611e6ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '775698af1ac0460194a1425eb62c8371', 'request_id': 'req-1d505806-8d4e-4893-a76c-3a5a7611e6ce'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:04.196188 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-1d505806-8d4e-4893-a76c-3a5a7611e6ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '775698af1ac0460194a1425eb62c8371', 'request_id': 'req-1d505806-8d4e-4893-a76c-3a5a7611e6ce'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:04.209561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-1d505806-8d4e-4893-a76c-3a5a7611e6ce admin admin] The token cache is being invalidate because project 775698af1ac0460194a1425eb62c8371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:04.209850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-1d505806-8d4e-4893-a76c-3a5a7611e6ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:04.210548 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-1d505806-8d4e-4893-a76c-3a5a7611e6ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:04.212156 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf375e89-e7ec-4a4d-a8fc-b14976193b8b None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:04.212276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1124/2261] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:04 2022] DELETE /identity/v3/projects/775698af1ac0460194a1425eb62c8371 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:04.218539 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf30f6-2117-4b30-aa5a-c4edba6fa642 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:04.218948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf30f6-2117-4b30-aa5a-c4edba6fa642 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:04.219233 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf30f6-2117-4b30-aa5a-c4edba6fa642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:04.239304 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-b0bf30f6-2117-4b30-aa5a-c4edba6fa642 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:04.240375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b0bf30f6-2117-4b30-aa5a-c4edba6fa642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:04.271118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a86cc-ff92-4a63-b80f-059f7c8cdbfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:04.271561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a86cc-ff92-4a63-b80f-059f7c8cdbfa service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:04.271844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a86cc-ff92-4a63-b80f-059f7c8cdbfa service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:04.272117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a86cc-ff92-4a63-b80f-059f7c8cdbfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:04.275759 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-255a86cc-ff92-4a63-b80f-059f7c8cdbfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:04.300858 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1125/2262] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:04.313747 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba124d-93dd-4eba-9e21-ff520b335067 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:04.314087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba124d-93dd-4eba-9e21-ff520b335067 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:04.314316 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba124d-93dd-4eba-9e21-ff520b335067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:04.332780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-afba124d-93dd-4eba-9e21-ff520b335067 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:04.334821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-afba124d-93dd-4eba-9e21-ff520b335067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:04.336854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a86cc-ff92-4a63-b80f-059f7c8cdbfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:04.338428 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a86cc-ff92-4a63-b80f-059f7c8cdbfa service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:04.344601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1126/2263] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:53:04.348951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1138/2264] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:53:04.349984 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba124d-93dd-4eba-9e21-ff520b335067 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:04.360634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba124d-93dd-4eba-9e21-ff520b335067 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:04.365629 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9e10d-10fb-4e87-b108-78b15a308270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:04.366284 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9e10d-10fb-4e87-b108-78b15a308270 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:04.366567 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9e10d-10fb-4e87-b108-78b15a308270 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:04.366877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9e10d-10fb-4e87-b108-78b15a308270 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:04.367633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd9e10d-10fb-4e87-b108-78b15a308270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:53:04.367938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd9e10d-10fb-4e87-b108-78b15a308270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:04.369605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd9e10d-10fb-4e87-b108-78b15a308270 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:04.381141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1127/2265] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:53:04.385689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a86cc-ff92-4a63-b80f-059f7c8cdbfa service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:04.400026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a86cc-ff92-4a63-b80f-059f7c8cdbfa service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:04.406084 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5b049-c604-4539-8128-f191d1e9d57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:04.406857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5b049-c604-4539-8128-f191d1e9d57b admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:04.407145 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5b049-c604-4539-8128-f191d1e9d57b admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:04.407425 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5b049-c604-4539-8128-f191d1e9d57b admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:04.408252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5b049-c604-4539-8128-f191d1e9d57b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:04.410886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5b049-c604-4539-8128-f191d1e9d57b admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:04.452847 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1139/2266] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:53:04 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:04.456819 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9e10d-10fb-4e87-b108-78b15a308270 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:04.471389 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9e10d-10fb-4e87-b108-78b15a308270 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:04.549207 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b229071-bfd2-47e7-b6d6-f92cc6f28689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:04.549897 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b229071-bfd2-47e7-b6d6-f92cc6f28689 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:04.550157 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b229071-bfd2-47e7-b6d6-f92cc6f28689 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:04.550434 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b229071-bfd2-47e7-b6d6-f92cc6f28689 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:04.551177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b229071-bfd2-47e7-b6d6-f92cc6f28689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:04.553111 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b229071-bfd2-47e7-b6d6-f92cc6f28689 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:04.577363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1128/2267] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:53:04 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:04.581206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5b049-c604-4539-8128-f191d1e9d57b admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:04.592406 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5b049-c604-4539-8128-f191d1e9d57b admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:04.597143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e3195-470f-4bd1-9601-b8412cc1d619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:04.597803 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e3195-470f-4bd1-9601-b8412cc1d619 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:04.598087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e3195-470f-4bd1-9601-b8412cc1d619 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:04.598367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e3195-470f-4bd1-9601-b8412cc1d619 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:04.599125 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e3195-470f-4bd1-9601-b8412cc1d619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:04.600734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e3195-470f-4bd1-9601-b8412cc1d619 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:04.611273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1140/2268] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:04.615613 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b229071-bfd2-47e7-b6d6-f92cc6f28689 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:04.627472 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b229071-bfd2-47e7-b6d6-f92cc6f28689 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:04.633306 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997d9f6-5389-4d0a-8bbd-10d74a76d78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:04.634037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997d9f6-5389-4d0a-8bbd-10d74a76d78d admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:04.634892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997d9f6-5389-4d0a-8bbd-10d74a76d78d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:04.634892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997d9f6-5389-4d0a-8bbd-10d74a76d78d admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:04.635528 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997d9f6-5389-4d0a-8bbd-10d74a76d78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:04.637194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997d9f6-5389-4d0a-8bbd-10d74a76d78d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:04.648471 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1129/2269] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:04.652670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e3195-470f-4bd1-9601-b8412cc1d619 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:04.663532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e3195-470f-4bd1-9601-b8412cc1d619 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:04.668234 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65350f04-c790-435e-946c-4e8b738a806e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:04.669138 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65350f04-c790-435e-946c-4e8b738a806e admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:04.669422 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65350f04-c790-435e-946c-4e8b738a806e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:04.669698 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65350f04-c790-435e-946c-4e8b738a806e admin admin] PATH_INFO: `/v3/projects/e5d482c1a50940479216a0a959deac5e/users/2633b2817d6f4c60b70a53abef9fcc0c/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:04.684012 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65350f04-c790-435e-946c-4e8b738a806e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d482c1a50940479216a0a959deac5e, user_id=2633b2817d6f4c60b70a53abef9fcc0c, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:04.686718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65350f04-c790-435e-946c-4e8b738a806e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:04.699117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1141/2270] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:04 2022] PUT /identity/v3/projects/e5d482c1a50940479216a0a959deac5e/users/2633b2817d6f4c60b70a53abef9fcc0c/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:04.709852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101dae7-6cf4-4299-ace8-3c9e37ed2cd0 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:04.710174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101dae7-6cf4-4299-ace8-3c9e37ed2cd0 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:04.710456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101dae7-6cf4-4299-ace8-3c9e37ed2cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:04.737720 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-9101dae7-6cf4-4299-ace8-3c9e37ed2cd0 None None] MFA Rules not processed for user `2633b2817d6f4c60b70a53abef9fcc0c`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:04.738981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-9101dae7-6cf4-4299-ace8-3c9e37ed2cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:04.801812 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1130/2271] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:05.473803 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65350f04-c790-435e-946c-4e8b738a806e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:05.476502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-65350f04-c790-435e-946c-4e8b738a806e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:05.537206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65350f04-c790-435e-946c-4e8b738a806e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:05.542885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cbbb9-72d1-4bf8-ad09-137c2d60a76e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:05.544933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cbbb9-72d1-4bf8-ad09-137c2d60a76e service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:05.545347 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cbbb9-72d1-4bf8-ad09-137c2d60a76e service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:05.545631 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cbbb9-72d1-4bf8-ad09-137c2d60a76e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:05.555277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cbbb9-72d1-4bf8-ad09-137c2d60a76e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:05.556841 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cbbb9-72d1-4bf8-ad09-137c2d60a76e service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:05.566654 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1142/2272] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:06.585757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101dae7-6cf4-4299-ace8-3c9e37ed2cd0 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:06.596816 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101dae7-6cf4-4299-ace8-3c9e37ed2cd0 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:06.600811 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205f5a6-ad17-4171-bbbf-3f93fc5a6c4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:06.601458 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205f5a6-ad17-4171-bbbf-3f93fc5a6c4b service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:06.601743 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205f5a6-ad17-4171-bbbf-3f93fc5a6c4b service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:06.602021 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205f5a6-ad17-4171-bbbf-3f93fc5a6c4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:06.603852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8205f5a6-ad17-4171-bbbf-3f93fc5a6c4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:06.675980 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205f5a6-ad17-4171-bbbf-3f93fc5a6c4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:06.677536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205f5a6-ad17-4171-bbbf-3f93fc5a6c4b service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:06.687978 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1131/2273] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:14.885409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cbbb9-72d1-4bf8-ad09-137c2d60a76e service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:14.887503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-5e4cbbb9-72d1-4bf8-ad09-137c2d60a76e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:15.008246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cbbb9-72d1-4bf8-ad09-137c2d60a76e service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:15.012874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c8ae3-e54a-45d3-86fb-3b403fedeeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:15.014334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c8ae3-e54a-45d3-86fb-3b403fedeeae admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:15.014617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c8ae3-e54a-45d3-86fb-3b403fedeeae admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:15.014925 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c8ae3-e54a-45d3-86fb-3b403fedeeae admin admin] PATH_INFO: `/v3/users/4d461c00682d4024911af8c38046cd75` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:15.016177 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c8ae3-e54a-45d3-86fb-3b403fedeeae admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d461c00682d4024911af8c38046cd75)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:15.018144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c8ae3-e54a-45d3-86fb-3b403fedeeae admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:15.068741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f90c8ae3-e54a-45d3-86fb-3b403fedeeae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d461c00682d4024911af8c38046cd75'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:15.079934 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f90c8ae3-e54a-45d3-86fb-3b403fedeeae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d461c00682d4024911af8c38046cd75'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:15.080588 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f90c8ae3-e54a-45d3-86fb-3b403fedeeae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d461c00682d4024911af8c38046cd75'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:15.081249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f90c8ae3-e54a-45d3-86fb-3b403fedeeae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d461c00682d4024911af8c38046cd75'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:15.087952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f90c8ae3-e54a-45d3-86fb-3b403fedeeae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d461c00682d4024911af8c38046cd75'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:15.100593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1143/2274] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:14 2022] DELETE /identity/v3/users/4d461c00682d4024911af8c38046cd75 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:15.105155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7175cf-0715-4458-9c63-883bf6a0c595 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:15.105641 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7175cf-0715-4458-9c63-883bf6a0c595 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:15.105751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7175cf-0715-4458-9c63-883bf6a0c595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:15.124526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-fd7175cf-0715-4458-9c63-883bf6a0c595 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:15.125749 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-fd7175cf-0715-4458-9c63-883bf6a0c595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:15.193048 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1132/2275] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:15.199413 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c8ae3-e54a-45d3-86fb-3b403fedeeae admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:15.200890 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f90c8ae3-e54a-45d3-86fb-3b403fedeeae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:15.258190 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c8ae3-e54a-45d3-86fb-3b403fedeeae admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:15.261965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731f1f0-29ea-4bd8-9864-3bc8eeaf8e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:15.262610 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731f1f0-29ea-4bd8-9864-3bc8eeaf8e40 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:15.262920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731f1f0-29ea-4bd8-9864-3bc8eeaf8e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:15.263223 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731f1f0-29ea-4bd8-9864-3bc8eeaf8e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:15.272535 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731f1f0-29ea-4bd8-9864-3bc8eeaf8e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:15.274000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731f1f0-29ea-4bd8-9864-3bc8eeaf8e40 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:15.284380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1144/2276] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:53:15.669877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7175cf-0715-4458-9c63-883bf6a0c595 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:15.671483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-fd7175cf-0715-4458-9c63-883bf6a0c595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:15.683104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7175cf-0715-4458-9c63-883bf6a0c595 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:15.687655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f877f70-fc51-46da-822c-bd7e1ca852ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:15.688508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f877f70-fc51-46da-822c-bd7e1ca852ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:15.688794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f877f70-fc51-46da-822c-bd7e1ca852ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:15.689068 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f877f70-fc51-46da-822c-bd7e1ca852ad admin admin] PATH_INFO: `/v3/projects/8fea379bf5c74e72b3b25a35f8327535` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:15.690238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f877f70-fc51-46da-822c-bd7e1ca852ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fea379bf5c74e72b3b25a35f8327535)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:15.692180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f877f70-fc51-46da-822c-bd7e1ca852ad admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:15.738608 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6f877f70-fc51-46da-822c-bd7e1ca852ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fea379bf5c74e72b3b25a35f8327535', 'request_id': 'req-6f877f70-fc51-46da-822c-bd7e1ca852ad'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:15.745285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6f877f70-fc51-46da-822c-bd7e1ca852ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fea379bf5c74e72b3b25a35f8327535', 'request_id': 'req-6f877f70-fc51-46da-822c-bd7e1ca852ad'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:15.752222 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6f877f70-fc51-46da-822c-bd7e1ca852ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fea379bf5c74e72b3b25a35f8327535', 'request_id': 'req-6f877f70-fc51-46da-822c-bd7e1ca852ad'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:15.762484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6f877f70-fc51-46da-822c-bd7e1ca852ad admin admin] The token cache is being invalidate because project 8fea379bf5c74e72b3b25a35f8327535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:15.762808 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6f877f70-fc51-46da-822c-bd7e1ca852ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:15.763974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6f877f70-fc51-46da-822c-bd7e1ca852ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:15.765545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1133/2277] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:15 2022] DELETE /identity/v3/projects/8fea379bf5c74e72b3b25a35f8327535 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:15.770262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca4854-95c4-4b10-9230-2e43d702e7e9 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:15.770549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca4854-95c4-4b10-9230-2e43d702e7e9 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:15.770857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca4854-95c4-4b10-9230-2e43d702e7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:15.788329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-5aca4854-95c4-4b10-9230-2e43d702e7e9 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:15.789516 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-5aca4854-95c4-4b10-9230-2e43d702e7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:15.845986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1145/2278] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:15.853703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386701c-6190-4135-842a-5cf5b7c62ae1 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:15.853997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386701c-6190-4135-842a-5cf5b7c62ae1 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:15.854280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386701c-6190-4135-842a-5cf5b7c62ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:15.872417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-2386701c-6190-4135-842a-5cf5b7c62ae1 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:15.873617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2386701c-6190-4135-842a-5cf5b7c62ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:15.881674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1134/2279] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:15.885789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca4854-95c4-4b10-9230-2e43d702e7e9 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:15.895851 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca4854-95c4-4b10-9230-2e43d702e7e9 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:15.900510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc386fa-b361-4f56-a740-7ccb188bf6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:15.901154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc386fa-b361-4f56-a740-7ccb188bf6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:15.901434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc386fa-b361-4f56-a740-7ccb188bf6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:15.901704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc386fa-b361-4f56-a740-7ccb188bf6f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:15.902419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc386fa-b361-4f56-a740-7ccb188bf6f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:53:15.902741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc386fa-b361-4f56-a740-7ccb188bf6f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:15.904881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc386fa-b361-4f56-a740-7ccb188bf6f8 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:15.919833 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1146/2280] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:53:15.920968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386701c-6190-4135-842a-5cf5b7c62ae1 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:15.930003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386701c-6190-4135-842a-5cf5b7c62ae1 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:15.934600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564ae76-c18a-4dee-aa79-64995b0aea0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:15.935301 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564ae76-c18a-4dee-aa79-64995b0aea0b admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:15.935619 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564ae76-c18a-4dee-aa79-64995b0aea0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:15.935842 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564ae76-c18a-4dee-aa79-64995b0aea0b admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:15.936526 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564ae76-c18a-4dee-aa79-64995b0aea0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:15.938438 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564ae76-c18a-4dee-aa79-64995b0aea0b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:15.967763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1135/2281] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:53:15 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:15.970746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc386fa-b361-4f56-a740-7ccb188bf6f8 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:15.981875 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc386fa-b361-4f56-a740-7ccb188bf6f8 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:16.045669 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158cf23d-dc0b-4250-9b1e-9e1c6462eb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:16.046363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cf23d-dc0b-4250-9b1e-9e1c6462eb56 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:16.046647 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cf23d-dc0b-4250-9b1e-9e1c6462eb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:16.046961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cf23d-dc0b-4250-9b1e-9e1c6462eb56 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:16.047711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158cf23d-dc0b-4250-9b1e-9e1c6462eb56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:16.049685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158cf23d-dc0b-4250-9b1e-9e1c6462eb56 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:16.069000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1147/2282] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:53:15 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:16.073431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564ae76-c18a-4dee-aa79-64995b0aea0b admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:16.084191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564ae76-c18a-4dee-aa79-64995b0aea0b admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:16.088892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a0830-58d1-424e-967e-f2c57c9f0f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:16.089605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a0830-58d1-424e-967e-f2c57c9f0f44 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:16.089954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a0830-58d1-424e-967e-f2c57c9f0f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:16.090298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a0830-58d1-424e-967e-f2c57c9f0f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:16.091200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a0830-58d1-424e-967e-f2c57c9f0f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:16.092878 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a0830-58d1-424e-967e-f2c57c9f0f44 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:16.103085 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1136/2283] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:16.107476 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158cf23d-dc0b-4250-9b1e-9e1c6462eb56 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:16.117677 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158cf23d-dc0b-4250-9b1e-9e1c6462eb56 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:16.122272 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398837b8-3f84-469d-b592-3c7a86dd9d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:16.123514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398837b8-3f84-469d-b592-3c7a86dd9d93 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:16.123795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398837b8-3f84-469d-b592-3c7a86dd9d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:16.124067 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398837b8-3f84-469d-b592-3c7a86dd9d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:16.124778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398837b8-3f84-469d-b592-3c7a86dd9d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:16.126373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398837b8-3f84-469d-b592-3c7a86dd9d93 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:16.140663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1148/2284] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:16.142004 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a0830-58d1-424e-967e-f2c57c9f0f44 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:16.152816 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a0830-58d1-424e-967e-f2c57c9f0f44 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:16.160306 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d46a2-aeb4-4cf6-8432-cfaa29818d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:16.161629 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d46a2-aeb4-4cf6-8432-cfaa29818d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:16.161987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d46a2-aeb4-4cf6-8432-cfaa29818d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:16.162349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d46a2-aeb4-4cf6-8432-cfaa29818d43 admin admin] PATH_INFO: `/v3/projects/7476c4e1892840df92f9a9dc382e85f9/users/f2ea289c76224c67a43267e86bc723a3/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:16.179302 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d46a2-aeb4-4cf6-8432-cfaa29818d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7476c4e1892840df92f9a9dc382e85f9, user_id=f2ea289c76224c67a43267e86bc723a3, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:16.181897 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d46a2-aeb4-4cf6-8432-cfaa29818d43 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:16.208242 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1137/2285] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:16 2022] PUT /identity/v3/projects/7476c4e1892840df92f9a9dc382e85f9/users/f2ea289c76224c67a43267e86bc723a3/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:16.219629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba7110-bf94-4aae-b23a-15a07aeeabe2 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:16.219925 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba7110-bf94-4aae-b23a-15a07aeeabe2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:16.220208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba7110-bf94-4aae-b23a-15a07aeeabe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:16.248051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-cfba7110-bf94-4aae-b23a-15a07aeeabe2 None None] MFA Rules not processed for user `f2ea289c76224c67a43267e86bc723a3`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:16.249288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-cfba7110-bf94-4aae-b23a-15a07aeeabe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:16.312258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1149/2286] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:16.323644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d46a2-aeb4-4cf6-8432-cfaa29818d43 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:16.325398 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-313d46a2-aeb4-4cf6-8432-cfaa29818d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:16.384711 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d46a2-aeb4-4cf6-8432-cfaa29818d43 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:16.388315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc10964-5964-4384-bd14-fa57925def43 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad57b046f3a43978d1c98bcc1a76d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:16.388947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc10964-5964-4384-bd14-fa57925def43 service glance] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:16.389228 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc10964-5964-4384-bd14-fa57925def43 service glance] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:16.389506 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc10964-5964-4384-bd14-fa57925def43 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:16.397499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc10964-5964-4384-bd14-fa57925def43 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:16.398953 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc10964-5964-4384-bd14-fa57925def43 service glance] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:16.408011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1138/2287] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:18.681760 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb7935-2e40-4a23-86bb-7a191d7bb14d None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:18.682235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb7935-2e40-4a23-86bb-7a191d7bb14d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:18.682326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb7935-2e40-4a23-86bb-7a191d7bb14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:18.701705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-fafb7935-2e40-4a23-86bb-7a191d7bb14d None None] MFA Rules not processed for user `004034ab6aca41ae893214bf0e81841b`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:18.702943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-fafb7935-2e40-4a23-86bb-7a191d7bb14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:18.758817 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1150/2288] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:38.062125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc10964-5964-4384-bd14-fa57925def43 service glance] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:38.063712 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-cbc10964-5964-4384-bd14-fa57925def43 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:38.132528 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc10964-5964-4384-bd14-fa57925def43 service glance] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:38.137138 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7cc55-e79f-405e-b94b-c307fa857f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:38.138003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7cc55-e79f-405e-b94b-c307fa857f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:38.138295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7cc55-e79f-405e-b94b-c307fa857f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:38.138576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7cc55-e79f-405e-b94b-c307fa857f4a admin admin] PATH_INFO: `/v3/users/f56effc6e229462882283f5bbf0bd209` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:38.140442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7cc55-e79f-405e-b94b-c307fa857f4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f56effc6e229462882283f5bbf0bd209)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:38.142405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7cc55-e79f-405e-b94b-c307fa857f4a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:38.188074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-88f7cc55-e79f-405e-b94b-c307fa857f4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f56effc6e229462882283f5bbf0bd209'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:38.200982 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-88f7cc55-e79f-405e-b94b-c307fa857f4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f56effc6e229462882283f5bbf0bd209'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:38.202012 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-88f7cc55-e79f-405e-b94b-c307fa857f4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f56effc6e229462882283f5bbf0bd209'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:38.203447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-88f7cc55-e79f-405e-b94b-c307fa857f4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f56effc6e229462882283f5bbf0bd209'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:38.211540 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-88f7cc55-e79f-405e-b94b-c307fa857f4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f56effc6e229462882283f5bbf0bd209'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:38.224330 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1139/2289] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:38 2022] DELETE /identity/v3/users/f56effc6e229462882283f5bbf0bd209 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:38.228885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c34e91-02bf-4d15-977f-78208b8eabc2 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:38.229408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c34e91-02bf-4d15-977f-78208b8eabc2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:38.229408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c34e91-02bf-4d15-977f-78208b8eabc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:38.248011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-29c34e91-02bf-4d15-977f-78208b8eabc2 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:38.250172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-29c34e91-02bf-4d15-977f-78208b8eabc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:38.310839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1151/2290] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:53:38.321221 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7cc55-e79f-405e-b94b-c307fa857f4a admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:38.322791 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-88f7cc55-e79f-405e-b94b-c307fa857f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:38.385584 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7cc55-e79f-405e-b94b-c307fa857f4a admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:38.389307 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6cd26-aee3-4e87-ba0d-f111f9ee43f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:38.390097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6cd26-aee3-4e87-ba0d-f111f9ee43f9 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:38.390424 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6cd26-aee3-4e87-ba0d-f111f9ee43f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:38.390726 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6cd26-aee3-4e87-ba0d-f111f9ee43f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:38.400478 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6cd26-aee3-4e87-ba0d-f111f9ee43f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:38.402029 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6cd26-aee3-4e87-ba0d-f111f9ee43f9 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:38.413794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1140/2291] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:38.700036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c34e91-02bf-4d15-977f-78208b8eabc2 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:38.701569 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-29c34e91-02bf-4d15-977f-78208b8eabc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:38.715489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c34e91-02bf-4d15-977f-78208b8eabc2 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:38.724879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9d92f-5255-40e2-85f8-2d55715bcd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:38.727029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9d92f-5255-40e2-85f8-2d55715bcd99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:38.727258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9d92f-5255-40e2-85f8-2d55715bcd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:38.727543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9d92f-5255-40e2-85f8-2d55715bcd99 admin admin] PATH_INFO: `/v3/projects/851dcca13cee48a19327e7bada45af31` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:38.728805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9d92f-5255-40e2-85f8-2d55715bcd99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=851dcca13cee48a19327e7bada45af31)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:38.730757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9d92f-5255-40e2-85f8-2d55715bcd99 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:38.785541 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a2b9d92f-5255-40e2-85f8-2d55715bcd99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '851dcca13cee48a19327e7bada45af31', 'request_id': 'req-a2b9d92f-5255-40e2-85f8-2d55715bcd99'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:38.793258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a2b9d92f-5255-40e2-85f8-2d55715bcd99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '851dcca13cee48a19327e7bada45af31', 'request_id': 'req-a2b9d92f-5255-40e2-85f8-2d55715bcd99'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:38.813580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a2b9d92f-5255-40e2-85f8-2d55715bcd99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '851dcca13cee48a19327e7bada45af31', 'request_id': 'req-a2b9d92f-5255-40e2-85f8-2d55715bcd99'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:38.833195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a2b9d92f-5255-40e2-85f8-2d55715bcd99 admin admin] The token cache is being invalidate because project 851dcca13cee48a19327e7bada45af31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:38.833503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a2b9d92f-5255-40e2-85f8-2d55715bcd99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:38.834549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a2b9d92f-5255-40e2-85f8-2d55715bcd99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:38.836913 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1152/2292] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:38 2022] DELETE /identity/v3/projects/851dcca13cee48a19327e7bada45af31 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:38.840987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6cd26-aee3-4e87-ba0d-f111f9ee43f9 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:38.851437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6cd26-aee3-4e87-ba0d-f111f9ee43f9 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:38.925778 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543476e-5c6e-4f9e-aff5-6ee1759aeb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:38.926674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543476e-5c6e-4f9e-aff5-6ee1759aeb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:38.927017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543476e-5c6e-4f9e-aff5-6ee1759aeb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:38.927293 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543476e-5c6e-4f9e-aff5-6ee1759aeb1b admin admin] PATH_INFO: `/v3/users/f417d7c9ad294948b2774846f9a1d239` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:38.929893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543476e-5c6e-4f9e-aff5-6ee1759aeb1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f417d7c9ad294948b2774846f9a1d239)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:38.930908 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543476e-5c6e-4f9e-aff5-6ee1759aeb1b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:39.009079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9543476e-5c6e-4f9e-aff5-6ee1759aeb1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f417d7c9ad294948b2774846f9a1d239'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:39.030011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9543476e-5c6e-4f9e-aff5-6ee1759aeb1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f417d7c9ad294948b2774846f9a1d239'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:39.031611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9543476e-5c6e-4f9e-aff5-6ee1759aeb1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f417d7c9ad294948b2774846f9a1d239'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:39.033541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9543476e-5c6e-4f9e-aff5-6ee1759aeb1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f417d7c9ad294948b2774846f9a1d239'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:39.046093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9543476e-5c6e-4f9e-aff5-6ee1759aeb1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f417d7c9ad294948b2774846f9a1d239'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:39.067756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1141/2293] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:38 2022] DELETE /identity/v3/users/f417d7c9ad294948b2774846f9a1d239 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:39.221728 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c905f4-6125-495f-a7d9-944f5b39d258 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:39.221728 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c905f4-6125-495f-a7d9-944f5b39d258 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:39.222019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c905f4-6125-495f-a7d9-944f5b39d258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:39.244308 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-35c905f4-6125-495f-a7d9-944f5b39d258 None None] MFA Rules not processed for user `c3a0de9ca12746be893b97d3cd11b826`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:39.245578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-35c905f4-6125-495f-a7d9-944f5b39d258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:39.303954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1153/2294] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:39.445908 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543476e-5c6e-4f9e-aff5-6ee1759aeb1b admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:39.447330 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-9543476e-5c6e-4f9e-aff5-6ee1759aeb1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:39.526856 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543476e-5c6e-4f9e-aff5-6ee1759aeb1b admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:39.532932 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb89ee-2e92-4392-85e2-8c93aa85602e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:39.533798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb89ee-2e92-4392-85e2-8c93aa85602e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:39.534111 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb89ee-2e92-4392-85e2-8c93aa85602e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:39.534360 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb89ee-2e92-4392-85e2-8c93aa85602e admin admin] PATH_INFO: `/v3/projects/c241599785234c62af723fe561c669c6` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:39.535903 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb89ee-2e92-4392-85e2-8c93aa85602e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c241599785234c62af723fe561c669c6)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:39.538019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb89ee-2e92-4392-85e2-8c93aa85602e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:39.593733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-22eb89ee-2e92-4392-85e2-8c93aa85602e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c241599785234c62af723fe561c669c6', 'request_id': 'req-22eb89ee-2e92-4392-85e2-8c93aa85602e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:39.600077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-22eb89ee-2e92-4392-85e2-8c93aa85602e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c241599785234c62af723fe561c669c6', 'request_id': 'req-22eb89ee-2e92-4392-85e2-8c93aa85602e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:39.608975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-22eb89ee-2e92-4392-85e2-8c93aa85602e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c241599785234c62af723fe561c669c6', 'request_id': 'req-22eb89ee-2e92-4392-85e2-8c93aa85602e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:39.620132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-22eb89ee-2e92-4392-85e2-8c93aa85602e admin admin] The token cache is being invalidate because project c241599785234c62af723fe561c669c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:39.620474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-22eb89ee-2e92-4392-85e2-8c93aa85602e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:39.621118 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-22eb89ee-2e92-4392-85e2-8c93aa85602e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:39.623472 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1142/2295] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:39 2022] DELETE /identity/v3/projects/c241599785234c62af723fe561c669c6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:39.631162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a524973-04af-4830-b017-974bf0c3ebe4 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:39.631442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a524973-04af-4830-b017-974bf0c3ebe4 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:39.631823 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a524973-04af-4830-b017-974bf0c3ebe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:39.652440 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-9a524973-04af-4830-b017-974bf0c3ebe4 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:39.653653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9a524973-04af-4830-b017-974bf0c3ebe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:39.720340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1154/2296] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:39.727949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e3f97-fd43-4a48-9c69-1ed93d228283 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:39.728230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e3f97-fd43-4a48-9c69-1ed93d228283 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:39.728505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e3f97-fd43-4a48-9c69-1ed93d228283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:39.748670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5d1e3f97-fd43-4a48-9c69-1ed93d228283 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:39.750541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5d1e3f97-fd43-4a48-9c69-1ed93d228283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:39.763076 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1143/2297] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:39.769484 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a524973-04af-4830-b017-974bf0c3ebe4 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:39.781860 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a524973-04af-4830-b017-974bf0c3ebe4 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:39.788527 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3da5d6-1391-46b4-93c1-7ec77d9173ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:39.789528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3da5d6-1391-46b4-93c1-7ec77d9173ee admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:39.789528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3da5d6-1391-46b4-93c1-7ec77d9173ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:39.789731 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3da5d6-1391-46b4-93c1-7ec77d9173ee admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:39.790499 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3da5d6-1391-46b4-93c1-7ec77d9173ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:53:39.790905 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3da5d6-1391-46b4-93c1-7ec77d9173ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:39.792528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3da5d6-1391-46b4-93c1-7ec77d9173ee admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:39.810922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1155/2298] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:53:39.817178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e3f97-fd43-4a48-9c69-1ed93d228283 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:39.826436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e3f97-fd43-4a48-9c69-1ed93d228283 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:39.832990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0c002-0cb7-4eec-915b-31f0d21fc002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:39.833695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0c002-0cb7-4eec-915b-31f0d21fc002 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:39.833979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0c002-0cb7-4eec-915b-31f0d21fc002 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:39.834255 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0c002-0cb7-4eec-915b-31f0d21fc002 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:39.834994 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0c002-0cb7-4eec-915b-31f0d21fc002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:39.837549 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0c002-0cb7-4eec-915b-31f0d21fc002 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:39.869147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1144/2299] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:53:39 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:39.873726 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3da5d6-1391-46b4-93c1-7ec77d9173ee admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:39.884453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3da5d6-1391-46b4-93c1-7ec77d9173ee admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:39.949636 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc21ee9-a6ad-4a39-ba26-89f5d2b7c603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:39.950301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc21ee9-a6ad-4a39-ba26-89f5d2b7c603 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:39.950582 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc21ee9-a6ad-4a39-ba26-89f5d2b7c603 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:39.950889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc21ee9-a6ad-4a39-ba26-89f5d2b7c603 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:39.951571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc21ee9-a6ad-4a39-ba26-89f5d2b7c603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:39.953523 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc21ee9-a6ad-4a39-ba26-89f5d2b7c603 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:39.973702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1156/2300] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:53:39 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:39.977475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0c002-0cb7-4eec-915b-31f0d21fc002 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:39.988541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0c002-0cb7-4eec-915b-31f0d21fc002 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:39.993571 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a432e-cbb4-4b58-8d86-343a46a82910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:39.994333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a432e-cbb4-4b58-8d86-343a46a82910 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:39.994691 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a432e-cbb4-4b58-8d86-343a46a82910 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:39.995062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a432e-cbb4-4b58-8d86-343a46a82910 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:39.995866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a432e-cbb4-4b58-8d86-343a46a82910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:39.997546 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a432e-cbb4-4b58-8d86-343a46a82910 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:40.008819 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1145/2301] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:40.012818 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc21ee9-a6ad-4a39-ba26-89f5d2b7c603 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:40.026390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc21ee9-a6ad-4a39-ba26-89f5d2b7c603 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:40.033278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ca7c2-288b-494d-a0cf-f848088b5275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:40.034068 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ca7c2-288b-494d-a0cf-f848088b5275 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:40.034862 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ca7c2-288b-494d-a0cf-f848088b5275 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:40.035227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ca7c2-288b-494d-a0cf-f848088b5275 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:40.037441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ca7c2-288b-494d-a0cf-f848088b5275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:40.039257 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ca7c2-288b-494d-a0cf-f848088b5275 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:40.056758 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1157/2302] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:40.060804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a432e-cbb4-4b58-8d86-343a46a82910 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:40.072784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a432e-cbb4-4b58-8d86-343a46a82910 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:40.078581 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b05cc-d48d-4b15-8157-d05923d50e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:40.079511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b05cc-d48d-4b15-8157-d05923d50e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:40.079792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b05cc-d48d-4b15-8157-d05923d50e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:40.080060 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b05cc-d48d-4b15-8157-d05923d50e35 admin admin] PATH_INFO: `/v3/projects/7c7f76dac57840e0afb886414adf2436/users/2d33398436be44a88133f6b249c82e71/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:40.098110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b05cc-d48d-4b15-8157-d05923d50e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c7f76dac57840e0afb886414adf2436, user_id=2d33398436be44a88133f6b249c82e71, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:40.100873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b05cc-d48d-4b15-8157-d05923d50e35 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:40.124002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1146/2303] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:40 2022] PUT /identity/v3/projects/7c7f76dac57840e0afb886414adf2436/users/2d33398436be44a88133f6b249c82e71/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:40.134926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757b351-793e-4d26-8e17-f8edde6ccf93 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:40.135216 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757b351-793e-4d26-8e17-f8edde6ccf93 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:40.135506 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757b351-793e-4d26-8e17-f8edde6ccf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:40.170283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-9757b351-793e-4d26-8e17-f8edde6ccf93 None None] MFA Rules not processed for user `2d33398436be44a88133f6b249c82e71`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:40.172391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9757b351-793e-4d26-8e17-f8edde6ccf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:40.282032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1158/2304] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:40.284392 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b05cc-d48d-4b15-8157-d05923d50e35 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:40.286343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0e3b05cc-d48d-4b15-8157-d05923d50e35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:40.290775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757b351-793e-4d26-8e17-f8edde6ccf93 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:40.490127 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b05cc-d48d-4b15-8157-d05923d50e35 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:40.496661 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610d2f3-1173-442e-9a54-626e5cd5fa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:40.502580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610d2f3-1173-442e-9a54-626e5cd5fa36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:40.504293 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610d2f3-1173-442e-9a54-626e5cd5fa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:40.506212 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757b351-793e-4d26-8e17-f8edde6ccf93 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:40.506384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610d2f3-1173-442e-9a54-626e5cd5fa36 admin admin] PATH_INFO: `/v3/users/2633b2817d6f4c60b70a53abef9fcc0c` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:40.509517 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610d2f3-1173-442e-9a54-626e5cd5fa36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2633b2817d6f4c60b70a53abef9fcc0c)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:40.512794 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610d2f3-1173-442e-9a54-626e5cd5fa36 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:40.514989 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95221f5-7a45-409c-bdba-5ee2753db484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:40.515806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95221f5-7a45-409c-bdba-5ee2753db484 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:40.518872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95221f5-7a45-409c-bdba-5ee2753db484 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:40.519380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95221f5-7a45-409c-bdba-5ee2753db484 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:40.533463 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95221f5-7a45-409c-bdba-5ee2753db484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:40.535832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95221f5-7a45-409c-bdba-5ee2753db484 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:40.599669 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1159/2305] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:53:40 2022] POST /identity/v3/projects => generated 387 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:40.604598 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95221f5-7a45-409c-bdba-5ee2753db484 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:40.607109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6610d2f3-1173-442e-9a54-626e5cd5fa36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2633b2817d6f4c60b70a53abef9fcc0c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:40.620899 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6610d2f3-1173-442e-9a54-626e5cd5fa36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2633b2817d6f4c60b70a53abef9fcc0c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:40.621651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6610d2f3-1173-442e-9a54-626e5cd5fa36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2633b2817d6f4c60b70a53abef9fcc0c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:40.622487 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6610d2f3-1173-442e-9a54-626e5cd5fa36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2633b2817d6f4c60b70a53abef9fcc0c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:40.630551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6610d2f3-1173-442e-9a54-626e5cd5fa36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2633b2817d6f4c60b70a53abef9fcc0c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:40.672512 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1147/2306] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:40 2022] DELETE /identity/v3/users/2633b2817d6f4c60b70a53abef9fcc0c => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:40.679497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81181a-8d87-4e64-afb5-e964d5715b29 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:40.679818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81181a-8d87-4e64-afb5-e964d5715b29 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:40.680083 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81181a-8d87-4e64-afb5-e964d5715b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:40.702551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ab81181a-8d87-4e64-afb5-e964d5715b29 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:40.703941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ab81181a-8d87-4e64-afb5-e964d5715b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:40.747930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95221f5-7a45-409c-bdba-5ee2753db484 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:40.816576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1148/2307] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:40.830434 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81181a-8d87-4e64-afb5-e964d5715b29 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:40.831969 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ab81181a-8d87-4e64-afb5-e964d5715b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:40.849977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe8d8c-3d09-4989-b9b0-33423f0bf379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:40.851040 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe8d8c-3d09-4989-b9b0-33423f0bf379 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:40.851271 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe8d8c-3d09-4989-b9b0-33423f0bf379 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:40.851640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe8d8c-3d09-4989-b9b0-33423f0bf379 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:40.853704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe8d8c-3d09-4989-b9b0-33423f0bf379 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:40.855872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe8d8c-3d09-4989-b9b0-33423f0bf379 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:40.882937 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1160/2308] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:53:40 2022] POST /identity/v3/users => generated 300 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:40.887155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe8d8c-3d09-4989-b9b0-33423f0bf379 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:40.890253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-29fe8d8c-3d09-4989-b9b0-33423f0bf379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:40.906090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe8d8c-3d09-4989-b9b0-33423f0bf379 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:40.909356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81181a-8d87-4e64-afb5-e964d5715b29 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:40.914365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5dfb44-aede-45a8-aaf6-0b2108e43ad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:40.915456 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e1687-0566-4abe-9390-78ff5d127d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:40.915608 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dfb44-aede-45a8-aaf6-0b2108e43ad2 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:40.915803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dfb44-aede-45a8-aaf6-0b2108e43ad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:40.916077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dfb44-aede-45a8-aaf6-0b2108e43ad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:40.917296 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e1687-0566-4abe-9390-78ff5d127d58 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:40.917762 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e1687-0566-4abe-9390-78ff5d127d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:40.918213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e1687-0566-4abe-9390-78ff5d127d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:40.919185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e1687-0566-4abe-9390-78ff5d127d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:40.921365 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e1687-0566-4abe-9390-78ff5d127d58 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:40.927891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5dfb44-aede-45a8-aaf6-0b2108e43ad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:40.929359 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5dfb44-aede-45a8-aaf6-0b2108e43ad2 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:40.937962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1161/2309] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:40.939336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e1687-0566-4abe-9390-78ff5d127d58 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:40.947547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1149/2310] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:53:40.950404 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e1687-0566-4abe-9390-78ff5d127d58 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:40.956979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cef05-6bd5-45bd-9542-3bb1db7fcbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:40.957436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cef05-6bd5-45bd-9542-3bb1db7fcbea admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:40.957724 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cef05-6bd5-45bd-9542-3bb1db7fcbea admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:40.958089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cef05-6bd5-45bd-9542-3bb1db7fcbea admin admin] PATH_INFO: `/v3/projects/a37f5877988a4aa08600b7bd3ab56558/users/33f3d35011aa468ab8cf0ef33d9d9735/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:40.975154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cef05-6bd5-45bd-9542-3bb1db7fcbea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37f5877988a4aa08600b7bd3ab56558, user_id=33f3d35011aa468ab8cf0ef33d9d9735, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:40.978049 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cef05-6bd5-45bd-9542-3bb1db7fcbea admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:41.008090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1162/2311] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:40 2022] PUT /identity/v3/projects/a37f5877988a4aa08600b7bd3ab56558/users/33f3d35011aa468ab8cf0ef33d9d9735/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:41.009652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5dfb44-aede-45a8-aaf6-0b2108e43ad2 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:41.022846 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5dfb44-aede-45a8-aaf6-0b2108e43ad2 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:41.105224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491608ab-cb33-461b-bcaa-1539bb804f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:41.106598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491608ab-cb33-461b-bcaa-1539bb804f7f admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:41.107363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491608ab-cb33-461b-bcaa-1539bb804f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:41.107796 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491608ab-cb33-461b-bcaa-1539bb804f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:41.109004 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491608ab-cb33-461b-bcaa-1539bb804f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:41.111187 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491608ab-cb33-461b-bcaa-1539bb804f7f admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:41.122671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1150/2312] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:41.127515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cef05-6bd5-45bd-9542-3bb1db7fcbea admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:41.138012 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cef05-6bd5-45bd-9542-3bb1db7fcbea admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:41.143588 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c830a1-a18a-4572-8aba-e5d35d2b9285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:41.144393 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c830a1-a18a-4572-8aba-e5d35d2b9285 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:41.144677 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c830a1-a18a-4572-8aba-e5d35d2b9285 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:41.144954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c830a1-a18a-4572-8aba-e5d35d2b9285 admin admin] PATH_INFO: `/v3/projects/a37f5877988a4aa08600b7bd3ab56558/users/33f3d35011aa468ab8cf0ef33d9d9735/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:41.146988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c830a1-a18a-4572-8aba-e5d35d2b9285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37f5877988a4aa08600b7bd3ab56558, user_id=33f3d35011aa468ab8cf0ef33d9d9735, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:41.150780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c830a1-a18a-4572-8aba-e5d35d2b9285 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:41.164667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1163/2313] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:41 2022] PUT /identity/v3/projects/a37f5877988a4aa08600b7bd3ab56558/users/33f3d35011aa468ab8cf0ef33d9d9735/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:41.175193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc85fe2-b921-44eb-a371-e0e485907323 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:41.175659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc85fe2-b921-44eb-a371-e0e485907323 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:41.176066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc85fe2-b921-44eb-a371-e0e485907323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:41.222507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5bc85fe2-b921-44eb-a371-e0e485907323 None None] MFA Rules not processed for user `33f3d35011aa468ab8cf0ef33d9d9735`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:41.223831 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5bc85fe2-b921-44eb-a371-e0e485907323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:41.304208 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1151/2314] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:41.315662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c830a1-a18a-4572-8aba-e5d35d2b9285 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:41.317555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-81c830a1-a18a-4572-8aba-e5d35d2b9285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:41.335530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc85fe2-b921-44eb-a371-e0e485907323 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:41.337155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5bc85fe2-b921-44eb-a371-e0e485907323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:41.394419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c830a1-a18a-4572-8aba-e5d35d2b9285 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:41.398102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2094f53-c4fb-4db2-9f74-e6971902bcf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:41.398894 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2094f53-c4fb-4db2-9f74-e6971902bcf8 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:41.399181 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2094f53-c4fb-4db2-9f74-e6971902bcf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:41.399458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2094f53-c4fb-4db2-9f74-e6971902bcf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:41.409554 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc85fe2-b921-44eb-a371-e0e485907323 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:41.410281 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2094f53-c4fb-4db2-9f74-e6971902bcf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:41.411999 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2094f53-c4fb-4db2-9f74-e6971902bcf8 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:41.414807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:41.415670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:41.416104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:41.416398 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] PATH_INFO: `/v3/projects/e5d482c1a50940479216a0a959deac5e` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:41.418072 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d482c1a50940479216a0a959deac5e)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:41.420444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:41.427763 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1164/2315] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:41.469010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d482c1a50940479216a0a959deac5e', 'request_id': 'req-05d0cef6-2932-4890-9357-29b813ea42db'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.475124 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d482c1a50940479216a0a959deac5e', 'request_id': 'req-05d0cef6-2932-4890-9357-29b813ea42db'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.483930 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d482c1a50940479216a0a959deac5e', 'request_id': 'req-05d0cef6-2932-4890-9357-29b813ea42db'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.500591 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] The token cache is being invalidate because project e5d482c1a50940479216a0a959deac5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:41.501066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.501792 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.503479 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1152/2316] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:41 2022] DELETE /identity/v3/projects/e5d482c1a50940479216a0a959deac5e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:53:41.508426 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9eaa-86bb-4cf8-ab48-6c0e05713895 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:41.508805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9eaa-86bb-4cf8-ab48-6c0e05713895 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:41.509293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9eaa-86bb-4cf8-ab48-6c0e05713895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:41.533212 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-02cb9eaa-86bb-4cf8-ab48-6c0e05713895 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:41.534826 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-02cb9eaa-86bb-4cf8-ab48-6c0e05713895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:41.568482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:41.570315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:41.634439 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1165/2317] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:41.641386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb9eaa-86bb-4cf8-ab48-6c0e05713895 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:41.641386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-02cb9eaa-86bb-4cf8-ab48-6c0e05713895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:41.652560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb9eaa-86bb-4cf8-ab48-6c0e05713895 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:41.654615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0cef6-2932-4890-9357-29b813ea42db admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:41.660154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae6cda-65f3-48c3-bea0-8aa167eaa7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:41.660890 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae6cda-65f3-48c3-bea0-8aa167eaa7f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:41.661353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae6cda-65f3-48c3-bea0-8aa167eaa7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:41.661795 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae6cda-65f3-48c3-bea0-8aa167eaa7f5 admin admin] PATH_INFO: `/v3/users/8b43b811255a43aa99597c809e1fb453` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:41.663264 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae6cda-65f3-48c3-bea0-8aa167eaa7f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b43b811255a43aa99597c809e1fb453)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:41.664118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004dea5-9503-4915-aec4-fc263027dac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:41.665432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004dea5-9503-4915-aec4-fc263027dac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:41.665819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004dea5-9503-4915-aec4-fc263027dac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:41.665965 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae6cda-65f3-48c3-bea0-8aa167eaa7f5 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:41.666544 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004dea5-9503-4915-aec4-fc263027dac6 admin admin] PATH_INFO: `/v3/users/2d33398436be44a88133f6b249c82e71` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:41.667811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004dea5-9503-4915-aec4-fc263027dac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d33398436be44a88133f6b249c82e71)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:41.670598 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004dea5-9503-4915-aec4-fc263027dac6 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:41.748845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a004dea5-9503-4915-aec4-fc263027dac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d33398436be44a88133f6b249c82e71'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.766089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a004dea5-9503-4915-aec4-fc263027dac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d33398436be44a88133f6b249c82e71'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.767501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a004dea5-9503-4915-aec4-fc263027dac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d33398436be44a88133f6b249c82e71'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.768141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a004dea5-9503-4915-aec4-fc263027dac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d33398436be44a88133f6b249c82e71'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.776684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a004dea5-9503-4915-aec4-fc263027dac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d33398436be44a88133f6b249c82e71'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.781406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ceae6cda-65f3-48c3-bea0-8aa167eaa7f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b43b811255a43aa99597c809e1fb453'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.802226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1166/2318] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:41 2022] DELETE /identity/v3/users/2d33398436be44a88133f6b249c82e71 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:41.802226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2f4e0-6887-4458-984d-e4115d17fd5a None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:41.802226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2f4e0-6887-4458-984d-e4115d17fd5a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:41.802226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2f4e0-6887-4458-984d-e4115d17fd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:41.809261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ceae6cda-65f3-48c3-bea0-8aa167eaa7f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b43b811255a43aa99597c809e1fb453'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.809995 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ceae6cda-65f3-48c3-bea0-8aa167eaa7f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b43b811255a43aa99597c809e1fb453'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.810681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ceae6cda-65f3-48c3-bea0-8aa167eaa7f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b43b811255a43aa99597c809e1fb453'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.827585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ceae6cda-65f3-48c3-bea0-8aa167eaa7f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b43b811255a43aa99597c809e1fb453'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:41.852901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-ddd2f4e0-6887-4458-984d-e4115d17fd5a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:41.854742 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ddd2f4e0-6887-4458-984d-e4115d17fd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:41.856992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1153/2319] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:41 2022] DELETE /identity/v3/users/8b43b811255a43aa99597c809e1fb453 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:53:41.859392 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db564b-7c6e-4c66-8e15-c8266f38931a None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:41.859827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db564b-7c6e-4c66-8e15-c8266f38931a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:41.860152 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db564b-7c6e-4c66-8e15-c8266f38931a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:41.881547 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-25db564b-7c6e-4c66-8e15-c8266f38931a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:41.882884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-25db564b-7c6e-4c66-8e15-c8266f38931a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:41.952404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1154/2320] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:41.957423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e20685-78dd-4ffe-bc40-5c4a1fcb22c5 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:41.957631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e20685-78dd-4ffe-bc40-5c4a1fcb22c5 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:41.957883 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e20685-78dd-4ffe-bc40-5c4a1fcb22c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:41.963165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1167/2321] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:41.967119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2f4e0-6887-4458-984d-e4115d17fd5a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:41.969325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ddd2f4e0-6887-4458-984d-e4115d17fd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:41.977974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-02e20685-78dd-4ffe-bc40-5c4a1fcb22c5 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:41.979354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-02e20685-78dd-4ffe-bc40-5c4a1fcb22c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:41.990749 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1155/2322] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:41.990749 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e20685-78dd-4ffe-bc40-5c4a1fcb22c5 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:42.009570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e20685-78dd-4ffe-bc40-5c4a1fcb22c5 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:42.014941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87b799-dd94-4846-8e3a-9c58bcd2c5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:42.015604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87b799-dd94-4846-8e3a-9c58bcd2c5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:42.017390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87b799-dd94-4846-8e3a-9c58bcd2c5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:42.017390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87b799-dd94-4846-8e3a-9c58bcd2c5b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:42.017390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87b799-dd94-4846-8e3a-9c58bcd2c5b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:53:42.017390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87b799-dd94-4846-8e3a-9c58bcd2c5b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:42.018876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87b799-dd94-4846-8e3a-9c58bcd2c5b8 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:42.031299 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1156/2323] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:53:42.032130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87b799-dd94-4846-8e3a-9c58bcd2c5b8 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:42.033949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2f87b799-dd94-4846-8e3a-9c58bcd2c5b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:42.053497 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2f4e0-6887-4458-984d-e4115d17fd5a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:42.056975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4508185-dbc1-4376-9105-1ee2f3fd0aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:42.057617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4508185-dbc1-4376-9105-1ee2f3fd0aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:42.057895 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4508185-dbc1-4376-9105-1ee2f3fd0aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:42.058167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4508185-dbc1-4376-9105-1ee2f3fd0aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:42.071326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4508185-dbc1-4376-9105-1ee2f3fd0aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:42.072796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4508185-dbc1-4376-9105-1ee2f3fd0aa9 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:42.083347 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1168/2324] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:42.085239 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4508185-dbc1-4376-9105-1ee2f3fd0aa9 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:42.094486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4508185-dbc1-4376-9105-1ee2f3fd0aa9 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:42.100994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a4eb0-4b3a-4f9c-bfa6-2100b35f6b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:42.101649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a4eb0-4b3a-4f9c-bfa6-2100b35f6b67 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:42.101932 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a4eb0-4b3a-4f9c-bfa6-2100b35f6b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:42.102165 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87b799-dd94-4846-8e3a-9c58bcd2c5b8 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:42.102333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a4eb0-4b3a-4f9c-bfa6-2100b35f6b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:42.102920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a4eb0-4b3a-4f9c-bfa6-2100b35f6b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:42.104965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a4eb0-4b3a-4f9c-bfa6-2100b35f6b67 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:42.111307 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47ed5e-1188-4708-ad74-afe67fa22c57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:42.112214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47ed5e-1188-4708-ad74-afe67fa22c57 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:42.112447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47ed5e-1188-4708-ad74-afe67fa22c57 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:42.112733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47ed5e-1188-4708-ad74-afe67fa22c57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:42.124414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47ed5e-1188-4708-ad74-afe67fa22c57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:42.125959 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47ed5e-1188-4708-ad74-afe67fa22c57 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:42.142115 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1169/2325] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:53:42 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:42.143854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1157/2326] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:42.147862 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a4eb0-4b3a-4f9c-bfa6-2100b35f6b67 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:42.237364 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a4eb0-4b3a-4f9c-bfa6-2100b35f6b67 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:42.243111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893f1ab-6e62-4375-8a41-6203dfad69c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:42.243809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893f1ab-6e62-4375-8a41-6203dfad69c0 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:42.244081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893f1ab-6e62-4375-8a41-6203dfad69c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:42.244357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893f1ab-6e62-4375-8a41-6203dfad69c0 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:42.245062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893f1ab-6e62-4375-8a41-6203dfad69c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:42.247073 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893f1ab-6e62-4375-8a41-6203dfad69c0 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:42.272326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1170/2327] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:53:42 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:42.278238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47ed5e-1188-4708-ad74-afe67fa22c57 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:42.319951 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47ed5e-1188-4708-ad74-afe67fa22c57 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:42.334595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f4e84-9a03-4d21-ae6f-aec3106c4cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:42.335465 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f4e84-9a03-4d21-ae6f-aec3106c4cda admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:42.335756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f4e84-9a03-4d21-ae6f-aec3106c4cda admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:42.336109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f4e84-9a03-4d21-ae6f-aec3106c4cda admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:42.336913 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f4e84-9a03-4d21-ae6f-aec3106c4cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:42.338767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f4e84-9a03-4d21-ae6f-aec3106c4cda admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:42.355558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1158/2328] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:42.356715 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893f1ab-6e62-4375-8a41-6203dfad69c0 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:42.371699 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893f1ab-6e62-4375-8a41-6203dfad69c0 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:42.379358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe8ecc-3751-44f7-a7ea-fca69d5aa04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:42.380082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe8ecc-3751-44f7-a7ea-fca69d5aa04e admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:42.380364 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe8ecc-3751-44f7-a7ea-fca69d5aa04e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:42.380638 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe8ecc-3751-44f7-a7ea-fca69d5aa04e admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:42.381709 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe8ecc-3751-44f7-a7ea-fca69d5aa04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:42.384127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe8ecc-3751-44f7-a7ea-fca69d5aa04e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:42.401862 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1171/2329] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:42.402045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f4e84-9a03-4d21-ae6f-aec3106c4cda admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:42.423438 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f4e84-9a03-4d21-ae6f-aec3106c4cda admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:42.430501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a47dc2-de9f-4b7f-bbc9-322bcab7219e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:42.431812 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a47dc2-de9f-4b7f-bbc9-322bcab7219e admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:42.432225 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a47dc2-de9f-4b7f-bbc9-322bcab7219e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:42.432690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a47dc2-de9f-4b7f-bbc9-322bcab7219e admin admin] PATH_INFO: `/v3/projects/2cda036215bb472daf8c7c9050a19fa8/users/a5020beed83b44a7b742cddaa9131fb7/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:42.487713 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a47dc2-de9f-4b7f-bbc9-322bcab7219e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cda036215bb472daf8c7c9050a19fa8, user_id=a5020beed83b44a7b742cddaa9131fb7, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:42.510769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a47dc2-de9f-4b7f-bbc9-322bcab7219e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:42.536013 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1159/2330] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:42 2022] PUT /identity/v3/projects/2cda036215bb472daf8c7c9050a19fa8/users/a5020beed83b44a7b742cddaa9131fb7/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:53:42.546891 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688497b-3a9f-4c1c-ade5-99f5007bc1b9 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:42.547184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688497b-3a9f-4c1c-ade5-99f5007bc1b9 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:42.547458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688497b-3a9f-4c1c-ade5-99f5007bc1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:42.589342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a688497b-3a9f-4c1c-ade5-99f5007bc1b9 None None] MFA Rules not processed for user `a5020beed83b44a7b742cddaa9131fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:42.591056 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a688497b-3a9f-4c1c-ade5-99f5007bc1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:42.684850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1172/2331] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:42.689962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a47dc2-de9f-4b7f-bbc9-322bcab7219e admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:42.707522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688497b-3a9f-4c1c-ade5-99f5007bc1b9 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:42.709668 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a688497b-3a9f-4c1c-ade5-99f5007bc1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:42.802086 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a47dc2-de9f-4b7f-bbc9-322bcab7219e admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:42.805002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688497b-3a9f-4c1c-ade5-99f5007bc1b9 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:42.807585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94009237-d229-45df-9850-2731cbb0cab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:42.808077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94009237-d229-45df-9850-2731cbb0cab9 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:42.808462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94009237-d229-45df-9850-2731cbb0cab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:42.808842 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94009237-d229-45df-9850-2731cbb0cab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:42.809672 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94009237-d229-45df-9850-2731cbb0cab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:42.812440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94009237-d229-45df-9850-2731cbb0cab9 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:42.813902 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:42.814951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:42.815321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:42.815615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] PATH_INFO: `/v3/projects/7c7f76dac57840e0afb886414adf2436` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:42.817037 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c7f76dac57840e0afb886414adf2436)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:42.819977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:42.862975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1160/2332] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:53:42 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:42.864085 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94009237-d229-45df-9850-2731cbb0cab9 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:42.871203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-94009237-d229-45df-9850-2731cbb0cab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:42.921724 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c7f76dac57840e0afb886414adf2436', 'request_id': 'req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:42.932702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c7f76dac57840e0afb886414adf2436', 'request_id': 'req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:42.947926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c7f76dac57840e0afb886414adf2436', 'request_id': 'req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:42.965552 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] The token cache is being invalidate because project 7c7f76dac57840e0afb886414adf2436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:42.965844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:42.967205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:42.968985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1173/2333] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:42 2022] DELETE /identity/v3/projects/7c7f76dac57840e0afb886414adf2436 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:53:42.969873 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:42.971737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:43.009601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94009237-d229-45df-9850-2731cbb0cab9 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:43.123550 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dbc4d-ff3d-469f-9bf5-c76e4e5eab89 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:43.146570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4ccb0-fbd3-42a8-954c-260f1470a001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:43.147291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4ccb0-fbd3-42a8-954c-260f1470a001 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:43.147579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4ccb0-fbd3-42a8-954c-260f1470a001 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:43.147854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4ccb0-fbd3-42a8-954c-260f1470a001 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:43.148556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4ccb0-fbd3-42a8-954c-260f1470a001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:43.155839 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c3184-6a22-4dcf-8912-82f97e8ef45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:43.156748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4ccb0-fbd3-42a8-954c-260f1470a001 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:43.157769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c3184-6a22-4dcf-8912-82f97e8ef45c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:43.158273 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c3184-6a22-4dcf-8912-82f97e8ef45c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:43.158895 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c3184-6a22-4dcf-8912-82f97e8ef45c admin admin] PATH_INFO: `/v3/projects/7690caec8c7c43b59b83e94ae25bfc4e` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:43.160491 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c3184-6a22-4dcf-8912-82f97e8ef45c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7690caec8c7c43b59b83e94ae25bfc4e)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:43.163686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c3184-6a22-4dcf-8912-82f97e8ef45c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:43.197904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1174/2334] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:53:42 2022] POST /identity/v3/users => generated 292 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:43.198865 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4ccb0-fbd3-42a8-954c-260f1470a001 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:43.200683 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-caf4ccb0-fbd3-42a8-954c-260f1470a001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:43.220570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4ccb0-fbd3-42a8-954c-260f1470a001 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:43.247331 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c61c3184-6a22-4dcf-8912-82f97e8ef45c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7690caec8c7c43b59b83e94ae25bfc4e', 'request_id': 'req-c61c3184-6a22-4dcf-8912-82f97e8ef45c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.261404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c61c3184-6a22-4dcf-8912-82f97e8ef45c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7690caec8c7c43b59b83e94ae25bfc4e', 'request_id': 'req-c61c3184-6a22-4dcf-8912-82f97e8ef45c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.281624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c61c3184-6a22-4dcf-8912-82f97e8ef45c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7690caec8c7c43b59b83e94ae25bfc4e', 'request_id': 'req-c61c3184-6a22-4dcf-8912-82f97e8ef45c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.311298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c61c3184-6a22-4dcf-8912-82f97e8ef45c admin admin] The token cache is being invalidate because project 7690caec8c7c43b59b83e94ae25bfc4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:43.313620 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c61c3184-6a22-4dcf-8912-82f97e8ef45c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.314655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c61c3184-6a22-4dcf-8912-82f97e8ef45c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.322366 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1161/2335] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:42 2022] DELETE /identity/v3/projects/7690caec8c7c43b59b83e94ae25bfc4e => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:43.323811 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89270b6-b252-4eef-9754-f288aa7efc6b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:43.324109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89270b6-b252-4eef-9754-f288aa7efc6b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:43.324394 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89270b6-b252-4eef-9754-f288aa7efc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:43.337178 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c35e50-6199-42c6-bafd-dfb19487cc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:43.338119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c35e50-6199-42c6-bafd-dfb19487cc69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:43.338399 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c35e50-6199-42c6-bafd-dfb19487cc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:43.338834 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c35e50-6199-42c6-bafd-dfb19487cc69 admin admin] PATH_INFO: `/v3/users/33f3d35011aa468ab8cf0ef33d9d9735` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:43.340869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c35e50-6199-42c6-bafd-dfb19487cc69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33f3d35011aa468ab8cf0ef33d9d9735)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:43.343503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c35e50-6199-42c6-bafd-dfb19487cc69 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:43.392990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-f89270b6-b252-4eef-9754-f288aa7efc6b None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:43.394636 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f89270b6-b252-4eef-9754-f288aa7efc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:43.426414 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-87c35e50-6199-42c6-bafd-dfb19487cc69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33f3d35011aa468ab8cf0ef33d9d9735'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.450313 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-87c35e50-6199-42c6-bafd-dfb19487cc69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33f3d35011aa468ab8cf0ef33d9d9735'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.451187 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-87c35e50-6199-42c6-bafd-dfb19487cc69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33f3d35011aa468ab8cf0ef33d9d9735'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.452640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-87c35e50-6199-42c6-bafd-dfb19487cc69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33f3d35011aa468ab8cf0ef33d9d9735'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.461398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-87c35e50-6199-42c6-bafd-dfb19487cc69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33f3d35011aa468ab8cf0ef33d9d9735'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.479265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1175/2336] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:43 2022] DELETE /identity/v3/users/33f3d35011aa468ab8cf0ef33d9d9735 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:43.481102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c35e50-6199-42c6-bafd-dfb19487cc69 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:43.483940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-87c35e50-6199-42c6-bafd-dfb19487cc69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:43.508391 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1162/2337] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:53:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:43.509877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89270b6-b252-4eef-9754-f288aa7efc6b None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:43.511884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f89270b6-b252-4eef-9754-f288aa7efc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:43.567047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89270b6-b252-4eef-9754-f288aa7efc6b None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:43.605204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c35e50-6199-42c6-bafd-dfb19487cc69 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:43.624278 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff21b79-bd16-4d33-a91d-3c388f2607ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:43.625136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff21b79-bd16-4d33-a91d-3c388f2607ca admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:43.625538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff21b79-bd16-4d33-a91d-3c388f2607ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:43.625860 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff21b79-bd16-4d33-a91d-3c388f2607ca admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:43.626758 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff21b79-bd16-4d33-a91d-3c388f2607ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:43.629908 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff21b79-bd16-4d33-a91d-3c388f2607ca admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:43.644722 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1176/2338] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:43.645772 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff21b79-bd16-4d33-a91d-3c388f2607ca admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:43.650935 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-fff21b79-bd16-4d33-a91d-3c388f2607ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:43.688787 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063ee6a-d8d3-4e45-b180-6d62ddefc6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:43.690155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063ee6a-d8d3-4e45-b180-6d62ddefc6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:43.690620 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063ee6a-d8d3-4e45-b180-6d62ddefc6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:43.691074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063ee6a-d8d3-4e45-b180-6d62ddefc6ab admin admin] PATH_INFO: `/v3/users/9363165e1cd54b47a11fb59b0c8b6b67` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:43.692493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063ee6a-d8d3-4e45-b180-6d62ddefc6ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=9363165e1cd54b47a11fb59b0c8b6b67)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:43.695363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063ee6a-d8d3-4e45-b180-6d62ddefc6ab admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:43.769526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff21b79-bd16-4d33-a91d-3c388f2607ca admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:43.777068 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27294a-2ad3-4cd1-9ae8-cb5a2466d448 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:43.777726 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27294a-2ad3-4cd1-9ae8-cb5a2466d448 service swift] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:43.778005 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27294a-2ad3-4cd1-9ae8-cb5a2466d448 service swift] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:43.778279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27294a-2ad3-4cd1-9ae8-cb5a2466d448 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:43.778853 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5063ee6a-d8d3-4e45-b180-6d62ddefc6ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9363165e1cd54b47a11fb59b0c8b6b67'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.797108 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5063ee6a-d8d3-4e45-b180-6d62ddefc6ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9363165e1cd54b47a11fb59b0c8b6b67'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.797974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5063ee6a-d8d3-4e45-b180-6d62ddefc6ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9363165e1cd54b47a11fb59b0c8b6b67'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.798638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5063ee6a-d8d3-4e45-b180-6d62ddefc6ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9363165e1cd54b47a11fb59b0c8b6b67'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.806678 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5063ee6a-d8d3-4e45-b180-6d62ddefc6ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9363165e1cd54b47a11fb59b0c8b6b67'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:43.824697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1163/2339] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:43 2022] DELETE /identity/v3/users/9363165e1cd54b47a11fb59b0c8b6b67 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:43.825809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063ee6a-d8d3-4e45-b180-6d62ddefc6ab admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:43.827748 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5063ee6a-d8d3-4e45-b180-6d62ddefc6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:43.890247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27294a-2ad3-4cd1-9ae8-cb5a2466d448 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:43.892036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27294a-2ad3-4cd1-9ae8-cb5a2466d448 service swift] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:43.904934 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1177/2340] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:53:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:53:43.914943 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063ee6a-d8d3-4e45-b180-6d62ddefc6ab admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:43.922444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99439bf0-fba6-4957-8257-efd9b79929a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:43.940125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99439bf0-fba6-4957-8257-efd9b79929a2 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:43.947716 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99439bf0-fba6-4957-8257-efd9b79929a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:43.948291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99439bf0-fba6-4957-8257-efd9b79929a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:43.949115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99439bf0-fba6-4957-8257-efd9b79929a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:43.953852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99439bf0-fba6-4957-8257-efd9b79929a2 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:43.996279 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1164/2341] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:44.002503 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27294a-2ad3-4cd1-9ae8-cb5a2466d448 service swift] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:44.017490 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27294a-2ad3-4cd1-9ae8-cb5a2466d448 service swift] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:44.024206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7c3cd-443d-4f36-b6f9-710ce7d302e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:44.025326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7c3cd-443d-4f36-b6f9-710ce7d302e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:44.025715 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7c3cd-443d-4f36-b6f9-710ce7d302e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:44.026487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7c3cd-443d-4f36-b6f9-710ce7d302e3 admin admin] PATH_INFO: `/v3/projects/489500d32bd347b8a49cb3a68befadc4/users/462398cf8e6f4ffaaa27da0986e236d4/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:44.033227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99439bf0-fba6-4957-8257-efd9b79929a2 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:44.038048 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-99439bf0-fba6-4957-8257-efd9b79929a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:44.073351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99439bf0-fba6-4957-8257-efd9b79929a2 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:44.103090 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:44.105139 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:44.108734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7c3cd-443d-4f36-b6f9-710ce7d302e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=489500d32bd347b8a49cb3a68befadc4, user_id=462398cf8e6f4ffaaa27da0986e236d4, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:44.109340 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:44.109826 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8 admin admin] PATH_INFO: `/v3/projects/a37f5877988a4aa08600b7bd3ab56558` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:44.113981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a37f5877988a4aa08600b7bd3ab56558)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:44.118964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7c3cd-443d-4f36-b6f9-710ce7d302e3 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:44.119270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:44.148586 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1178/2342] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:44 2022] PUT /identity/v3/projects/489500d32bd347b8a49cb3a68befadc4/users/462398cf8e6f4ffaaa27da0986e236d4/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:44.159832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44a938-a47a-4a56-abcc-625bade90a65 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:44.160125 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44a938-a47a-4a56-abcc-625bade90a65 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:44.160543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44a938-a47a-4a56-abcc-625bade90a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:44.220136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-3c44a938-a47a-4a56-abcc-625bade90a65 None None] MFA Rules not processed for user `462398cf8e6f4ffaaa27da0986e236d4`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:44.221310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3c44a938-a47a-4a56-abcc-625bade90a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:44.224806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a37f5877988a4aa08600b7bd3ab56558', 'request_id': 'req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:44.240400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a37f5877988a4aa08600b7bd3ab56558', 'request_id': 'req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:44.261306 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a37f5877988a4aa08600b7bd3ab56558', 'request_id': 'req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:44.279092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8 admin admin] The token cache is being invalidate because project a37f5877988a4aa08600b7bd3ab56558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:44.279386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:44.281850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bf3f90d0-9918-418d-b4ba-a448cb6db1e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:44.283553 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1165/2343] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:44 2022] DELETE /identity/v3/projects/a37f5877988a4aa08600b7bd3ab56558 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:44.289730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2502570-9dca-4bee-88fe-af929fab6794 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:44.290035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2502570-9dca-4bee-88fe-af929fab6794 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:44.290314 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2502570-9dca-4bee-88fe-af929fab6794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:44.356544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-e2502570-9dca-4bee-88fe-af929fab6794 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:44.358573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-e2502570-9dca-4bee-88fe-af929fab6794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:44.439302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1179/2344] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:53:44.440930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44a938-a47a-4a56-abcc-625bade90a65 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:44.448181 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3c44a938-a47a-4a56-abcc-625bade90a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:44.543518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1166/2345] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:53:44.551942 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af37f5-4655-47b6-bd89-e665fb07ad4b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:44.552230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af37f5-4655-47b6-bd89-e665fb07ad4b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:44.552509 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af37f5-4655-47b6-bd89-e665fb07ad4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:44.574583 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-c5af37f5-4655-47b6-bd89-e665fb07ad4b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:44.576393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c5af37f5-4655-47b6-bd89-e665fb07ad4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:44.593773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44a938-a47a-4a56-abcc-625bade90a65 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:44.594806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1167/2346] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:53:44.599241 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af37f5-4655-47b6-bd89-e665fb07ad4b None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:44.601645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dd07e-83c6-47c3-9128-5d23062119dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:44.602337 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dd07e-83c6-47c3-9128-5d23062119dd admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:44.602623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dd07e-83c6-47c3-9128-5d23062119dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:44.602930 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dd07e-83c6-47c3-9128-5d23062119dd admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:44.603635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6dd07e-83c6-47c3-9128-5d23062119dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:44.605785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6dd07e-83c6-47c3-9128-5d23062119dd admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:44.621859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af37f5-4655-47b6-bd89-e665fb07ad4b None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:44.634069 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8e0c4-4eb9-4626-8093-c50d1ff1e606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:44.635135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8e0c4-4eb9-4626-8093-c50d1ff1e606 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:44.635710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8e0c4-4eb9-4626-8093-c50d1ff1e606 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:44.636182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8e0c4-4eb9-4626-8093-c50d1ff1e606 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:44.637209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8e0c4-4eb9-4626-8093-c50d1ff1e606 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:53:44.638250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8e0c4-4eb9-4626-8093-c50d1ff1e606 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:44.640760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8e0c4-4eb9-4626-8093-c50d1ff1e606 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:44.656720 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1168/2347] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:53:44.666510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1180/2348] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:53:44 2022] POST /identity/v3/projects => generated 385 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:44.676074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8e0c4-4eb9-4626-8093-c50d1ff1e606 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:44.676292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dd07e-83c6-47c3-9128-5d23062119dd admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:44.696920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8e0c4-4eb9-4626-8093-c50d1ff1e606 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:44.818551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32669a6-0cce-411b-a140-b5b7fccd526c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:44.819906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32669a6-0cce-411b-a140-b5b7fccd526c admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:44.820481 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32669a6-0cce-411b-a140-b5b7fccd526c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:44.821606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32669a6-0cce-411b-a140-b5b7fccd526c admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:44.822623 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32669a6-0cce-411b-a140-b5b7fccd526c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:44.823559 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dd07e-83c6-47c3-9128-5d23062119dd admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:44.825889 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32669a6-0cce-411b-a140-b5b7fccd526c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:44.831287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8331e2c-7664-4b57-8faf-66ef0a09eb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:44.831864 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8331e2c-7664-4b57-8faf-66ef0a09eb3a admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:44.832154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8331e2c-7664-4b57-8faf-66ef0a09eb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:44.832431 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8331e2c-7664-4b57-8faf-66ef0a09eb3a admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:44.833227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8331e2c-7664-4b57-8faf-66ef0a09eb3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:44.836084 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8331e2c-7664-4b57-8faf-66ef0a09eb3a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:44.885388 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1181/2349] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:53:44 2022] POST /identity/v3/users => generated 299 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:44.890760 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8331e2c-7664-4b57-8faf-66ef0a09eb3a admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:44.895347 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f8331e2c-7664-4b57-8faf-66ef0a09eb3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:44.897370 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1169/2350] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:53:44 2022] POST /identity/v3/projects => generated 373 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:44.898545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32669a6-0cce-411b-a140-b5b7fccd526c admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:44.917303 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8331e2c-7664-4b57-8faf-66ef0a09eb3a admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:45.009931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32669a6-0cce-411b-a140-b5b7fccd526c admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:45.010157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:45.010914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.011200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.011487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] PATH_INFO: `/v3/projects/1688f45086ca4cfdaf54863bc16ce590` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.012858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1688f45086ca4cfdaf54863bc16ce590)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:45.015643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:45.019686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b177e-af07-4069-91f9-c59ff0af8349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:45.031459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b177e-af07-4069-91f9-c59ff0af8349 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.035108 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b177e-af07-4069-91f9-c59ff0af8349 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.046815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b177e-af07-4069-91f9-c59ff0af8349 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.048831 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b177e-af07-4069-91f9-c59ff0af8349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:45.052660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b177e-af07-4069-91f9-c59ff0af8349 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:45.094785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1170/2351] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:45.096831 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b177e-af07-4069-91f9-c59ff0af8349 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:45.137301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1688f45086ca4cfdaf54863bc16ce590', 'request_id': 'req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:45.138243 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b177e-af07-4069-91f9-c59ff0af8349 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:45.165169 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1688f45086ca4cfdaf54863bc16ce590', 'request_id': 'req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:45.184011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1688f45086ca4cfdaf54863bc16ce590', 'request_id': 'req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:45.214228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] The token cache is being invalidate because project 1688f45086ca4cfdaf54863bc16ce590 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:45.214526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:45.215322 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:45.217068 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1182/2352] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:44 2022] DELETE /identity/v3/projects/1688f45086ca4cfdaf54863bc16ce590 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:45.222366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:45.222366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:45.318436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545de1d0-6d3c-49bc-894a-4ab8957b3290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:45.319354 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545de1d0-6d3c-49bc-894a-4ab8957b3290 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.319743 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545de1d0-6d3c-49bc-894a-4ab8957b3290 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.320121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545de1d0-6d3c-49bc-894a-4ab8957b3290 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.320946 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545de1d0-6d3c-49bc-894a-4ab8957b3290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:45.323148 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545de1d0-6d3c-49bc-894a-4ab8957b3290 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:45.353336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1171/2353] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:53:45 2022] POST /identity/v3/users => generated 287 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:45.355693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b63ebd-33f6-42c4-819e-cac1acb39976 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.356127 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b63ebd-33f6-42c4-819e-cac1acb39976 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.356805 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b63ebd-33f6-42c4-819e-cac1acb39976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.370046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cb570-4cf3-4b50-8034-6b0d91f7e10d admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:45.378220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901be064-7ceb-4ebc-9030-a0638a4bf94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:45.379662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901be064-7ceb-4ebc-9030-a0638a4bf94f admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.380143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901be064-7ceb-4ebc-9030-a0638a4bf94f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.380825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901be064-7ceb-4ebc-9030-a0638a4bf94f admin admin] PATH_INFO: `/v3/projects/c2a821f22e63499fb82018a3feb7ff9c/users/d4d6e7f065104df384724c70f633c27c/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.395997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-76b63ebd-33f6-42c4-819e-cac1acb39976 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:45.397419 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-76b63ebd-33f6-42c4-819e-cac1acb39976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:45.407988 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1172/2354] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:45.408702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b63ebd-33f6-42c4-819e-cac1acb39976 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:45.411202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-76b63ebd-33f6-42c4-819e-cac1acb39976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:45.420709 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901be064-7ceb-4ebc-9030-a0638a4bf94f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2a821f22e63499fb82018a3feb7ff9c, user_id=d4d6e7f065104df384724c70f633c27c, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:45.424209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901be064-7ceb-4ebc-9030-a0638a4bf94f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:45.425501 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b63ebd-33f6-42c4-819e-cac1acb39976 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:45.434409 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a4345-3c1b-4aad-b5f9-ddf6d9ecb52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:45.435206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4345-3c1b-4aad-b5f9-ddf6d9ecb52c admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.436002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4345-3c1b-4aad-b5f9-ddf6d9ecb52c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.436002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4345-3c1b-4aad-b5f9-ddf6d9ecb52c admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.437664 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a4345-3c1b-4aad-b5f9-ddf6d9ecb52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:45.439732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a4345-3c1b-4aad-b5f9-ddf6d9ecb52c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:45.456331 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1173/2355] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:45.457885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77324e77-e548-4ff0-855e-ba2053b31dfd None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.458233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1183/2356] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:45 2022] PUT /identity/v3/projects/c2a821f22e63499fb82018a3feb7ff9c/users/d4d6e7f065104df384724c70f633c27c/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:45.461451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77324e77-e548-4ff0-855e-ba2053b31dfd None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.461451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77324e77-e548-4ff0-855e-ba2053b31dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.462746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901be064-7ceb-4ebc-9030-a0638a4bf94f admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:45.494008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-77324e77-e548-4ff0-855e-ba2053b31dfd None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:45.496453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-77324e77-e548-4ff0-855e-ba2053b31dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:45.554119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901be064-7ceb-4ebc-9030-a0638a4bf94f admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:45.559323 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2124571-defe-4036-ae27-098137dda83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:45.560225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2124571-defe-4036-ae27-098137dda83c admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.560679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2124571-defe-4036-ae27-098137dda83c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.561794 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2124571-defe-4036-ae27-098137dda83c admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.562696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2124571-defe-4036-ae27-098137dda83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:45.564715 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2124571-defe-4036-ae27-098137dda83c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:45.576576 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1184/2357] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:45.578453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2124571-defe-4036-ae27-098137dda83c admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:45.589897 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2124571-defe-4036-ae27-098137dda83c admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:45.595417 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8db0d-1a30-4922-beb6-93a627a29b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:45.596190 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8db0d-1a30-4922-beb6-93a627a29b9f admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.596579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8db0d-1a30-4922-beb6-93a627a29b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.596962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8db0d-1a30-4922-beb6-93a627a29b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.597797 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8db0d-1a30-4922-beb6-93a627a29b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:45.599102 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1174/2358] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:45.599656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8db0d-1a30-4922-beb6-93a627a29b9f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:45.600717 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77324e77-e548-4ff0-855e-ba2053b31dfd None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:45.614812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1185/2359] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:45.615871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8db0d-1a30-4922-beb6-93a627a29b9f admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:45.626129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77324e77-e548-4ff0-855e-ba2053b31dfd None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:45.627390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8db0d-1a30-4922-beb6-93a627a29b9f admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:45.632575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c23219-9a25-46e9-91ed-85cd679ba944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:45.633453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c23219-9a25-46e9-91ed-85cd679ba944 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.633869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c23219-9a25-46e9-91ed-85cd679ba944 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.634549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c23219-9a25-46e9-91ed-85cd679ba944 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.635613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c23219-9a25-46e9-91ed-85cd679ba944 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:53:45.636742 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c23219-9a25-46e9-91ed-85cd679ba944 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:45.637827 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c23219-9a25-46e9-91ed-85cd679ba944 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:45.644219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78992e83-45e4-486f-8f15-fd86245101ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:45.645849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78992e83-45e4-486f-8f15-fd86245101ef admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.646331 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78992e83-45e4-486f-8f15-fd86245101ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.646978 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78992e83-45e4-486f-8f15-fd86245101ef admin admin] PATH_INFO: `/v3/projects/fe45eb5a29704f66a1efa9790cee0d12/users/44a72193f6084e87bcd90b18bca9c8bb/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.661617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1186/2360] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:53:45.662401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c23219-9a25-46e9-91ed-85cd679ba944 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:45.684189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78992e83-45e4-486f-8f15-fd86245101ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe45eb5a29704f66a1efa9790cee0d12, user_id=44a72193f6084e87bcd90b18bca9c8bb, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:45.688042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78992e83-45e4-486f-8f15-fd86245101ef admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:45.688336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c23219-9a25-46e9-91ed-85cd679ba944 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:45.699581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199df92-ec29-433d-a20a-5b378d933d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:45.701215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199df92-ec29-433d-a20a-5b378d933d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.701776 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199df92-ec29-433d-a20a-5b378d933d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.702933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199df92-ec29-433d-a20a-5b378d933d6b admin admin] PATH_INFO: `/v3/projects/c2a821f22e63499fb82018a3feb7ff9c/users/d4d6e7f065104df384724c70f633c27c/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.705449 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199df92-ec29-433d-a20a-5b378d933d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2a821f22e63499fb82018a3feb7ff9c, user_id=d4d6e7f065104df384724c70f633c27c, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:45.705676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1175/2361] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:45 2022] PUT /identity/v3/projects/fe45eb5a29704f66a1efa9790cee0d12/users/44a72193f6084e87bcd90b18bca9c8bb/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:45.706908 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78992e83-45e4-486f-8f15-fd86245101ef admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:45.709380 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199df92-ec29-433d-a20a-5b378d933d6b admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:45.737356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1187/2362] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:45 2022] PUT /identity/v3/projects/c2a821f22e63499fb82018a3feb7ff9c/users/d4d6e7f065104df384724c70f633c27c/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:45.738968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d496359-21f6-492e-b517-34aaa7daddbf None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.740537 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d496359-21f6-492e-b517-34aaa7daddbf None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.742130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d496359-21f6-492e-b517-34aaa7daddbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.779550 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-2d496359-21f6-492e-b517-34aaa7daddbf None None] MFA Rules not processed for user `44a72193f6084e87bcd90b18bca9c8bb`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:45.780786 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-2d496359-21f6-492e-b517-34aaa7daddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:45.816604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78992e83-45e4-486f-8f15-fd86245101ef admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:45.900250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1188/2363] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:45.905981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe53f5-8845-4599-a1b6-67d4a8bf7d3e None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.907250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe53f5-8845-4599-a1b6-67d4a8bf7d3e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.907751 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe53f5-8845-4599-a1b6-67d4a8bf7d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.926848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4ae91-dfcb-4f5e-aef5-b43c7baf2b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:45.928042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4ae91-dfcb-4f5e-aef5-b43c7baf2b56 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:45.928331 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4ae91-dfcb-4f5e-aef5-b43c7baf2b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:45.928616 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4ae91-dfcb-4f5e-aef5-b43c7baf2b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:45.929617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4ae91-dfcb-4f5e-aef5-b43c7baf2b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:45.931601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4ae91-dfcb-4f5e-aef5-b43c7baf2b56 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:45.959771 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-0fbe53f5-8845-4599-a1b6-67d4a8bf7d3e None None] MFA Rules not processed for user `d4d6e7f065104df384724c70f633c27c`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:45.961774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-0fbe53f5-8845-4599-a1b6-67d4a8bf7d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:45.994455 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1176/2364] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:53:45 2022] POST /identity/v3/projects => generated 365 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:45.995797 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4ae91-dfcb-4f5e-aef5-b43c7baf2b56 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:45.997739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a5b4ae91-dfcb-4f5e-aef5-b43c7baf2b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:46.099070 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1189/2365] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:46.100148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe53f5-8845-4599-a1b6-67d4a8bf7d3e None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:46.120778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe53f5-8845-4599-a1b6-67d4a8bf7d3e None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:46.190270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4ae91-dfcb-4f5e-aef5-b43c7baf2b56 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:46.194984 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c91fed-fe41-4910-ba9f-34b86a8e81e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:46.195638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c91fed-fe41-4910-ba9f-34b86a8e81e1 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:46.195921 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c91fed-fe41-4910-ba9f-34b86a8e81e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:46.196290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c91fed-fe41-4910-ba9f-34b86a8e81e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:46.224871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4765c32-b0f7-4123-ab23-0554974efa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:46.226019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4765c32-b0f7-4123-ab23-0554974efa48 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:46.226827 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4765c32-b0f7-4123-ab23-0554974efa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:46.227314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4765c32-b0f7-4123-ab23-0554974efa48 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:46.228222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4765c32-b0f7-4123-ab23-0554974efa48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:46.230386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4765c32-b0f7-4123-ab23-0554974efa48 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:46.264831 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1190/2366] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:53:46 2022] POST /identity/v3/users => generated 283 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:46.265738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4765c32-b0f7-4123-ab23-0554974efa48 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:46.267649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f4765c32-b0f7-4123-ab23-0554974efa48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:46.286911 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4765c32-b0f7-4123-ab23-0554974efa48 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:46.299683 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c91fed-fe41-4910-ba9f-34b86a8e81e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:46.301874 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c91fed-fe41-4910-ba9f-34b86a8e81e1 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:46.302318 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1763d2-1da7-4317-bf5e-2227a5857440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:46.303184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1763d2-1da7-4317-bf5e-2227a5857440 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:46.303634 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1763d2-1da7-4317-bf5e-2227a5857440 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:46.304080 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1763d2-1da7-4317-bf5e-2227a5857440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:46.306402 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ce1763d2-1da7-4317-bf5e-2227a5857440 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:46.320638 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1177/2367] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:46.321511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c91fed-fe41-4910-ba9f-34b86a8e81e1 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:46.346874 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c91fed-fe41-4910-ba9f-34b86a8e81e1 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:46.366011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2394e38-6179-4a0e-8e93-7888e19a1865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:46.366808 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2394e38-6179-4a0e-8e93-7888e19a1865 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:46.367169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2394e38-6179-4a0e-8e93-7888e19a1865 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:46.368112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2394e38-6179-4a0e-8e93-7888e19a1865 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:46.368910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2394e38-6179-4a0e-8e93-7888e19a1865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:46.371225 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2394e38-6179-4a0e-8e93-7888e19a1865 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:46.386617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1178/2368] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:46.390519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2394e38-6179-4a0e-8e93-7888e19a1865 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:46.403824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2394e38-6179-4a0e-8e93-7888e19a1865 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:46.410449 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092a7a2-6500-4d42-bdfd-5071d16b9cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:46.411477 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092a7a2-6500-4d42-bdfd-5071d16b9cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:46.411813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092a7a2-6500-4d42-bdfd-5071d16b9cef admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:46.412178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092a7a2-6500-4d42-bdfd-5071d16b9cef admin admin] PATH_INFO: `/v3/users/44a72193f6084e87bcd90b18bca9c8bb` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:46.413771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092a7a2-6500-4d42-bdfd-5071d16b9cef admin admin] RBAC: Authorizing `identity:delete_user(user_id=44a72193f6084e87bcd90b18bca9c8bb)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:46.415971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092a7a2-6500-4d42-bdfd-5071d16b9cef admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:46.435285 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1763d2-1da7-4317-bf5e-2227a5857440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:46.438102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1763d2-1da7-4317-bf5e-2227a5857440 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:46.452807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1191/2369] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:46.454248 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1763d2-1da7-4317-bf5e-2227a5857440 service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:46.470261 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1763d2-1da7-4317-bf5e-2227a5857440 service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:46.486237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5215354-a68c-4bb0-8466-beecd1e19770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:46.487210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5215354-a68c-4bb0-8466-beecd1e19770 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:46.488553 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5215354-a68c-4bb0-8466-beecd1e19770 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:46.491015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5215354-a68c-4bb0-8466-beecd1e19770 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:46.503802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5215354-a68c-4bb0-8466-beecd1e19770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:46.506095 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5215354-a68c-4bb0-8466-beecd1e19770 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:46.520736 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1192/2370] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:46.524748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5215354-a68c-4bb0-8466-beecd1e19770 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:46.537045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5215354-a68c-4bb0-8466-beecd1e19770 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:46.543713 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b746f-cfd3-40a0-917d-a18243cf49c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:46.545192 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b746f-cfd3-40a0-917d-a18243cf49c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:46.545613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b746f-cfd3-40a0-917d-a18243cf49c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:46.545997 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b746f-cfd3-40a0-917d-a18243cf49c6 admin admin] PATH_INFO: `/v3/projects/1350fbd6055e4a91b6cda001b6fa827a/users/9ec3411098944d6c9f2dc3a8a2696b8d/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:46.549071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f092a7a2-6500-4d42-bdfd-5071d16b9cef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44a72193f6084e87bcd90b18bca9c8bb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:46.567546 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b746f-cfd3-40a0-917d-a18243cf49c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1350fbd6055e4a91b6cda001b6fa827a, user_id=9ec3411098944d6c9f2dc3a8a2696b8d, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:46.570783 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b746f-cfd3-40a0-917d-a18243cf49c6 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:46.575295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f092a7a2-6500-4d42-bdfd-5071d16b9cef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44a72193f6084e87bcd90b18bca9c8bb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:46.577692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f092a7a2-6500-4d42-bdfd-5071d16b9cef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44a72193f6084e87bcd90b18bca9c8bb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:46.579037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f092a7a2-6500-4d42-bdfd-5071d16b9cef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44a72193f6084e87bcd90b18bca9c8bb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:46.594165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1193/2371] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:46 2022] PUT /identity/v3/projects/1350fbd6055e4a91b6cda001b6fa827a/users/9ec3411098944d6c9f2dc3a8a2696b8d/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:53:46.611875 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-f092a7a2-6500-4d42-bdfd-5071d16b9cef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44a72193f6084e87bcd90b18bca9c8bb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:46.612635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476052-5bf5-44db-93e0-7ee0aeeb4233 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:46.613249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476052-5bf5-44db-93e0-7ee0aeeb4233 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:46.613740 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476052-5bf5-44db-93e0-7ee0aeeb4233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:46.647204 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1179/2372] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:46 2022] DELETE /identity/v3/users/44a72193f6084e87bcd90b18bca9c8bb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:46.659609 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c1afe-ea1c-4fed-b57d-84794d8bdc5f None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:46.660474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c1afe-ea1c-4fed-b57d-84794d8bdc5f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:46.663019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c1afe-ea1c-4fed-b57d-84794d8bdc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:46.664497 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-88476052-5bf5-44db-93e0-7ee0aeeb4233 None None] MFA Rules not processed for user `9ec3411098944d6c9f2dc3a8a2696b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:46.665806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-88476052-5bf5-44db-93e0-7ee0aeeb4233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:46.706902 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-be3c1afe-ea1c-4fed-b57d-84794d8bdc5f None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:46.709132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-be3c1afe-ea1c-4fed-b57d-84794d8bdc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:46.826163 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1194/2373] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:46.831637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88476052-5bf5-44db-93e0-7ee0aeeb4233 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:46.833763 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-88476052-5bf5-44db-93e0-7ee0aeeb4233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:46.868184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1180/2374] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:46.877880 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c1afe-ea1c-4fed-b57d-84794d8bdc5f None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:46.880706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-be3c1afe-ea1c-4fed-b57d-84794d8bdc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:46.950962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88476052-5bf5-44db-93e0-7ee0aeeb4233 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:46.958519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb17b32-441e-4615-9406-2b8e2db3f349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:46.959280 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb17b32-441e-4615-9406-2b8e2db3f349 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:46.959561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb17b32-441e-4615-9406-2b8e2db3f349 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:46.959882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb17b32-441e-4615-9406-2b8e2db3f349 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:46.960784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb17b32-441e-4615-9406-2b8e2db3f349 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:46.963164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb17b32-441e-4615-9406-2b8e2db3f349 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:46.977734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c1afe-ea1c-4fed-b57d-84794d8bdc5f None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:46.983532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fdb66-2b8e-434b-a98e-e667a5dc9922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:46.984463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fdb66-2b8e-434b-a98e-e667a5dc9922 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:46.984845 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fdb66-2b8e-434b-a98e-e667a5dc9922 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:46.985334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fdb66-2b8e-434b-a98e-e667a5dc9922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:46.996690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fdb66-2b8e-434b-a98e-e667a5dc9922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:47.000436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fdb66-2b8e-434b-a98e-e667a5dc9922 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:47.003476 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1195/2375] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:53:46 2022] POST /identity/v3/projects => generated 367 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:47.007227 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb17b32-441e-4615-9406-2b8e2db3f349 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:47.019527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1181/2376] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:53:47.170234 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb17b32-441e-4615-9406-2b8e2db3f349 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:47.176090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854fef0-cadf-460b-923c-019e0c1aa71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:47.177114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854fef0-cadf-460b-923c-019e0c1aa71a admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:47.177637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854fef0-cadf-460b-923c-019e0c1aa71a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:47.178118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854fef0-cadf-460b-923c-019e0c1aa71a admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:47.179045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854fef0-cadf-460b-923c-019e0c1aa71a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:47.181240 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854fef0-cadf-460b-923c-019e0c1aa71a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:47.232292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1196/2377] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:53:47 2022] POST /identity/v3/users => generated 290 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:53:47.236905 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fdb66-2b8e-434b-a98e-e667a5dc9922 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:47.259752 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fdb66-2b8e-434b-a98e-e667a5dc9922 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:47.266938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe8d30-5237-4612-ace8-70f7dc17ab34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:47.267733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe8d30-5237-4612-ace8-70f7dc17ab34 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:47.284940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe8d30-5237-4612-ace8-70f7dc17ab34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:47.285704 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe8d30-5237-4612-ace8-70f7dc17ab34 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:47.289224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe8d30-5237-4612-ace8-70f7dc17ab34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:47.315855 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe8d30-5237-4612-ace8-70f7dc17ab34 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:47.342183 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1182/2378] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:47.345988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854fef0-cadf-460b-923c-019e0c1aa71a admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:47.359852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854fef0-cadf-460b-923c-019e0c1aa71a admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:47.367944 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ea658-aebe-41f4-a75d-fb37df4fde44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:47.369784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ea658-aebe-41f4-a75d-fb37df4fde44 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:47.370161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ea658-aebe-41f4-a75d-fb37df4fde44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:47.370761 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ea658-aebe-41f4-a75d-fb37df4fde44 admin admin] PATH_INFO: `/v3/projects/72138e97086b4d67bb462e2dfcb067ba/users/c00d95e8226e4388b505f28df799bc05/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:47.393034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ea658-aebe-41f4-a75d-fb37df4fde44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72138e97086b4d67bb462e2dfcb067ba, user_id=c00d95e8226e4388b505f28df799bc05, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:47.398071 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ea658-aebe-41f4-a75d-fb37df4fde44 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:47.419460 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1197/2379] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:47 2022] PUT /identity/v3/projects/72138e97086b4d67bb462e2dfcb067ba/users/c00d95e8226e4388b505f28df799bc05/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:47.423761 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe8d30-5237-4612-ace8-70f7dc17ab34 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:47.443294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe8d30-5237-4612-ace8-70f7dc17ab34 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:47.507534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ea658-aebe-41f4-a75d-fb37df4fde44 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:47.514004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-680ea658-aebe-41f4-a75d-fb37df4fde44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:47.613432 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143129a-c537-4852-a0cf-b7bb1e0bc019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:47.614212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143129a-c537-4852-a0cf-b7bb1e0bc019 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:47.614887 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143129a-c537-4852-a0cf-b7bb1e0bc019 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:47.615234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143129a-c537-4852-a0cf-b7bb1e0bc019 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:47.616181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143129a-c537-4852-a0cf-b7bb1e0bc019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:47.617870 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143129a-c537-4852-a0cf-b7bb1e0bc019 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:47.629321 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1183/2380] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:53:47.643075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143129a-c537-4852-a0cf-b7bb1e0bc019 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:47.644021 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ea658-aebe-41f4-a75d-fb37df4fde44 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:47.651967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c9991-5a35-4a37-80a8-65f695f97691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:47.652845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c9991-5a35-4a37-80a8-65f695f97691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:47.653125 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c9991-5a35-4a37-80a8-65f695f97691 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:47.653408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c9991-5a35-4a37-80a8-65f695f97691 admin admin] PATH_INFO: `/v3/projects/fe45eb5a29704f66a1efa9790cee0d12` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:47.654607 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c9991-5a35-4a37-80a8-65f695f97691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe45eb5a29704f66a1efa9790cee0d12)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:47.657251 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c9991-5a35-4a37-80a8-65f695f97691 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:47.667387 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143129a-c537-4852-a0cf-b7bb1e0bc019 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:47.685653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b181091-0ab8-4ebd-a41e-98be38e3c98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:47.688325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b181091-0ab8-4ebd-a41e-98be38e3c98d admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:47.689224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b181091-0ab8-4ebd-a41e-98be38e3c98d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:47.689733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b181091-0ab8-4ebd-a41e-98be38e3c98d admin admin] PATH_INFO: `/v3/projects/72138e97086b4d67bb462e2dfcb067ba/users/c00d95e8226e4388b505f28df799bc05/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:47.694716 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b181091-0ab8-4ebd-a41e-98be38e3c98d admin admin] RBAC: Authorizing `identity:create_grant(project_id=72138e97086b4d67bb462e2dfcb067ba, user_id=c00d95e8226e4388b505f28df799bc05, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:47.705962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b181091-0ab8-4ebd-a41e-98be38e3c98d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:47.753627 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1184/2381] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:47 2022] PUT /identity/v3/projects/72138e97086b4d67bb462e2dfcb067ba/users/c00d95e8226e4388b505f28df799bc05/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:47.764873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22102ef7-edca-438f-a544-3e1700e96ffd None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:47.766493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22102ef7-edca-438f-a544-3e1700e96ffd None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:47.766988 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22102ef7-edca-438f-a544-3e1700e96ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:47.787089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a24c9991-5a35-4a37-80a8-65f695f97691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe45eb5a29704f66a1efa9790cee0d12', 'request_id': 'req-a24c9991-5a35-4a37-80a8-65f695f97691'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:47.799070 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a24c9991-5a35-4a37-80a8-65f695f97691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe45eb5a29704f66a1efa9790cee0d12', 'request_id': 'req-a24c9991-5a35-4a37-80a8-65f695f97691'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:47.826945 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a24c9991-5a35-4a37-80a8-65f695f97691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe45eb5a29704f66a1efa9790cee0d12', 'request_id': 'req-a24c9991-5a35-4a37-80a8-65f695f97691'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:47.828481 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-22102ef7-edca-438f-a544-3e1700e96ffd None None] MFA Rules not processed for user `c00d95e8226e4388b505f28df799bc05`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:47.830555 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-22102ef7-edca-438f-a544-3e1700e96ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:47.839962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a24c9991-5a35-4a37-80a8-65f695f97691 admin admin] The token cache is being invalidate because project fe45eb5a29704f66a1efa9790cee0d12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:47.841358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a24c9991-5a35-4a37-80a8-65f695f97691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:47.843113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a24c9991-5a35-4a37-80a8-65f695f97691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:47.848662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1198/2382] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:47 2022] DELETE /identity/v3/projects/fe45eb5a29704f66a1efa9790cee0d12 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:47.854098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740af6e-a618-4b3c-a241-4dd104ef167d None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:47.854416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740af6e-a618-4b3c-a241-4dd104ef167d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:47.854676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740af6e-a618-4b3c-a241-4dd104ef167d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:47.891333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-4740af6e-a618-4b3c-a241-4dd104ef167d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:47.892986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-4740af6e-a618-4b3c-a241-4dd104ef167d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:47.934393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1185/2383] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:47 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:53:47.945643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22102ef7-edca-438f-a544-3e1700e96ffd None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:47.947089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-22102ef7-edca-438f-a544-3e1700e96ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:47.969527 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1199/2384] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:53:47.978582 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c2c7e-331e-45e4-82de-80fbca0972ba None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:47.979065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c2c7e-331e-45e4-82de-80fbca0972ba None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:47.979328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c2c7e-331e-45e4-82de-80fbca0972ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:48.001139 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-6c1c2c7e-331e-45e4-82de-80fbca0972ba None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:48.002522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-6c1c2c7e-331e-45e4-82de-80fbca0972ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:48.015032 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1200/2385] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:53:48.016094 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c2c7e-331e-45e4-82de-80fbca0972ba None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:48.026129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c2c7e-331e-45e4-82de-80fbca0972ba None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:48.033004 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1ab52-bd66-4057-bc8f-f2ab278fa622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:48.033615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1ab52-bd66-4057-bc8f-f2ab278fa622 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:48.033754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1ab52-bd66-4057-bc8f-f2ab278fa622 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:48.034136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1ab52-bd66-4057-bc8f-f2ab278fa622 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:48.035436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1ab52-bd66-4057-bc8f-f2ab278fa622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:53:48.035436 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1ab52-bd66-4057-bc8f-f2ab278fa622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:48.037302 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1ab52-bd66-4057-bc8f-f2ab278fa622 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:48.058874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1201/2386] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:53:48.060226 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1ab52-bd66-4057-bc8f-f2ab278fa622 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:48.072453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1ab52-bd66-4057-bc8f-f2ab278fa622 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:48.076216 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22102ef7-edca-438f-a544-3e1700e96ffd None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:48.078286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f62ba-55c6-4d7a-a365-64ddbd533dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:48.079301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f62ba-55c6-4d7a-a365-64ddbd533dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:48.079649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f62ba-55c6-4d7a-a365-64ddbd533dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:48.080119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f62ba-55c6-4d7a-a365-64ddbd533dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:48.080910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f62ba-55c6-4d7a-a365-64ddbd533dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:48.081572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feeba6-5847-4bfe-8a3b-1fb64bb80198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:48.082267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feeba6-5847-4bfe-8a3b-1fb64bb80198 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:48.082556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feeba6-5847-4bfe-8a3b-1fb64bb80198 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:48.082865 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feeba6-5847-4bfe-8a3b-1fb64bb80198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:48.083260 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f62ba-55c6-4d7a-a365-64ddbd533dd2 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:48.097085 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97feeba6-5847-4bfe-8a3b-1fb64bb80198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:48.098598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97feeba6-5847-4bfe-8a3b-1fb64bb80198 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:48.110819 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1186/2387] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:47 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:48.120547 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1202/2388] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:53:48 2022] POST /identity/v3/projects => generated 363 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:53:48.121363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feeba6-5847-4bfe-8a3b-1fb64bb80198 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:48.132767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feeba6-5847-4bfe-8a3b-1fb64bb80198 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:48.266609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f62ba-55c6-4d7a-a365-64ddbd533dd2 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:48.267245 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd75b9-2981-421d-babe-2479245c7007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:48.267929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd75b9-2981-421d-babe-2479245c7007 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:48.268215 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd75b9-2981-421d-babe-2479245c7007 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:48.268502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd75b9-2981-421d-babe-2479245c7007 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:48.269202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd75b9-2981-421d-babe-2479245c7007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:48.271289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd75b9-2981-421d-babe-2479245c7007 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:48.299353 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1187/2389] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:53:48 2022] POST /identity/v3/users => generated 282 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:53:48.301178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd75b9-2981-421d-babe-2479245c7007 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:48.311716 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd75b9-2981-421d-babe-2479245c7007 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:48.317528 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c971d-e9e9-4cb6-97df-294b9a1367fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:48.318363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c971d-e9e9-4cb6-97df-294b9a1367fe admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:48.318628 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c971d-e9e9-4cb6-97df-294b9a1367fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:48.319005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c971d-e9e9-4cb6-97df-294b9a1367fe admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:48.319825 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c971d-e9e9-4cb6-97df-294b9a1367fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:48.321527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c971d-e9e9-4cb6-97df-294b9a1367fe admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:48.356520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1188/2390] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:53:48.357919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c971d-e9e9-4cb6-97df-294b9a1367fe admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:48.369231 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c971d-e9e9-4cb6-97df-294b9a1367fe admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:48.375150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df2b39-eabd-4d08-818b-6ef3f0082b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:48.375935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df2b39-eabd-4d08-818b-6ef3f0082b25 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:48.376321 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df2b39-eabd-4d08-818b-6ef3f0082b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:48.376709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df2b39-eabd-4d08-818b-6ef3f0082b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:48.377102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f62ba-55c6-4d7a-a365-64ddbd533dd2 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:48.377631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df2b39-eabd-4d08-818b-6ef3f0082b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:48.379420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df2b39-eabd-4d08-818b-6ef3f0082b25 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:48.386825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24135f76-3f48-4547-873f-abfe031409e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:48.387985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24135f76-3f48-4547-873f-abfe031409e6 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:48.391219 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24135f76-3f48-4547-873f-abfe031409e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:48.392027 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24135f76-3f48-4547-873f-abfe031409e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:48.395083 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1189/2391] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:53:48.396020 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-24135f76-3f48-4547-873f-abfe031409e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:48.396167 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df2b39-eabd-4d08-818b-6ef3f0082b25 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:48.408157 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df2b39-eabd-4d08-818b-6ef3f0082b25 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:48.414334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c7d49-752d-41cb-b14d-f5d64969b212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:48.415455 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c7d49-752d-41cb-b14d-f5d64969b212 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:48.415732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c7d49-752d-41cb-b14d-f5d64969b212 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:48.416070 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c7d49-752d-41cb-b14d-f5d64969b212 admin admin] PATH_INFO: `/v3/projects/49e6f023ae0f420fb463780287b8caab/users/3737f6f4251f41c8af7a5bcb01a70776/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:48.444619 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c7d49-752d-41cb-b14d-f5d64969b212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e6f023ae0f420fb463780287b8caab, user_id=3737f6f4251f41c8af7a5bcb01a70776, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:48.448691 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c7d49-752d-41cb-b14d-f5d64969b212 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:48.497607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1190/2392] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:53:48 2022] PUT /identity/v3/projects/49e6f023ae0f420fb463780287b8caab/users/3737f6f4251f41c8af7a5bcb01a70776/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:48.509012 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7f336-e0bc-456e-89be-eda52d8bca2b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:48.509468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7f336-e0bc-456e-89be-eda52d8bca2b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:48.509754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7f336-e0bc-456e-89be-eda52d8bca2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:48.570609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24135f76-3f48-4547-873f-abfe031409e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:48.572739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24135f76-3f48-4547-873f-abfe031409e6 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:48.579258 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-a4b7f336-e0bc-456e-89be-eda52d8bca2b None None] MFA Rules not processed for user `3737f6f4251f41c8af7a5bcb01a70776`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:48.580478 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a4b7f336-e0bc-456e-89be-eda52d8bca2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:48.605567 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1203/2393] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:48.607400 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24135f76-3f48-4547-873f-abfe031409e6 service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:48.624485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-24135f76-3f48-4547-873f-abfe031409e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:48.656725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24135f76-3f48-4547-873f-abfe031409e6 service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:48.798675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1191/2394] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:48 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:53:48.816272 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f74ae8-32bd-4229-9e28-bf6b92da4a16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:48.817475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f74ae8-32bd-4229-9e28-bf6b92da4a16 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:48.817896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f74ae8-32bd-4229-9e28-bf6b92da4a16 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:48.818487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f74ae8-32bd-4229-9e28-bf6b92da4a16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:48.823899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c9f74ae8-32bd-4229-9e28-bf6b92da4a16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:48.956573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f74ae8-32bd-4229-9e28-bf6b92da4a16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:48.959730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f74ae8-32bd-4229-9e28-bf6b92da4a16 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:48.984739 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1204/2395] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:49.383662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7f336-e0bc-456e-89be-eda52d8bca2b None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:49.396318 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7f336-e0bc-456e-89be-eda52d8bca2b None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:49.401891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d704a-de9a-4e26-ad4a-3c3e875e0469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:49.402956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d704a-de9a-4e26-ad4a-3c3e875e0469 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:49.405685 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d704a-de9a-4e26-ad4a-3c3e875e0469 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:49.405967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d704a-de9a-4e26-ad4a-3c3e875e0469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:49.424653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d704a-de9a-4e26-ad4a-3c3e875e0469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:49.429014 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d704a-de9a-4e26-ad4a-3c3e875e0469 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:49.453452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1192/2396] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:49 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:50.600089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f74ae8-32bd-4229-9e28-bf6b92da4a16 service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:50.610239 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f74ae8-32bd-4229-9e28-bf6b92da4a16 service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:50.615333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a02342-34a0-43b5-947a-c40545f15e21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:50.616277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a02342-34a0-43b5-947a-c40545f15e21 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:50.616787 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a02342-34a0-43b5-947a-c40545f15e21 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:50.617250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a02342-34a0-43b5-947a-c40545f15e21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:50.619410 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-16a02342-34a0-43b5-947a-c40545f15e21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:50.720326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a02342-34a0-43b5-947a-c40545f15e21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:50.721830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a02342-34a0-43b5-947a-c40545f15e21 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:50.733355 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1205/2397] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:57.497733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d704a-de9a-4e26-ad4a-3c3e875e0469 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:57.499757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-827d704a-de9a-4e26-ad4a-3c3e875e0469 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:57.821256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d704a-de9a-4e26-ad4a-3c3e875e0469 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:57.827134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56378365-7e9c-461b-b1bb-e8cfda2f9c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:57.828140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56378365-7e9c-461b-b1bb-e8cfda2f9c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:57.828544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56378365-7e9c-461b-b1bb-e8cfda2f9c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:57.828891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56378365-7e9c-461b-b1bb-e8cfda2f9c4d admin admin] PATH_INFO: `/v3/users/a5020beed83b44a7b742cddaa9131fb7` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:57.830345 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56378365-7e9c-461b-b1bb-e8cfda2f9c4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5020beed83b44a7b742cddaa9131fb7)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:57.832470 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56378365-7e9c-461b-b1bb-e8cfda2f9c4d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:57.902360 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-56378365-7e9c-461b-b1bb-e8cfda2f9c4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5020beed83b44a7b742cddaa9131fb7'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:57.929032 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-56378365-7e9c-461b-b1bb-e8cfda2f9c4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5020beed83b44a7b742cddaa9131fb7'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:57.930592 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-56378365-7e9c-461b-b1bb-e8cfda2f9c4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5020beed83b44a7b742cddaa9131fb7'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:57.931270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-56378365-7e9c-461b-b1bb-e8cfda2f9c4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5020beed83b44a7b742cddaa9131fb7'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:57.944570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-56378365-7e9c-461b-b1bb-e8cfda2f9c4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5020beed83b44a7b742cddaa9131fb7'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:57.974536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1193/2398] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:57 2022] DELETE /identity/v3/users/a5020beed83b44a7b742cddaa9131fb7 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:57.976705 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92154dd-8493-40ef-a4d3-a8932772a8e2 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:57.977286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92154dd-8493-40ef-a4d3-a8932772a8e2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:57.977685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92154dd-8493-40ef-a4d3-a8932772a8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:58.032693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d92154dd-8493-40ef-a4d3-a8932772a8e2 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:58.034426 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d92154dd-8493-40ef-a4d3-a8932772a8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:58.144137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1206/2399] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:53:58.158380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56378365-7e9c-461b-b1bb-e8cfda2f9c4d admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:58.160372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-56378365-7e9c-461b-b1bb-e8cfda2f9c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:58.220844 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92154dd-8493-40ef-a4d3-a8932772a8e2 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:58.222453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d92154dd-8493-40ef-a4d3-a8932772a8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:58.242679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92154dd-8493-40ef-a4d3-a8932772a8e2 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:58.253618 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1403b3-916a-45e8-bec4-593ea19a39d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:58.253618 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1403b3-916a-45e8-bec4-593ea19a39d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:58.253618 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1403b3-916a-45e8-bec4-593ea19a39d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:58.253618 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1403b3-916a-45e8-bec4-593ea19a39d5 admin admin] PATH_INFO: `/v3/users/9ec3411098944d6c9f2dc3a8a2696b8d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:58.255145 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1403b3-916a-45e8-bec4-593ea19a39d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec3411098944d6c9f2dc3a8a2696b8d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:58.258454 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1403b3-916a-45e8-bec4-593ea19a39d5 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:58.320874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7b1403b3-916a-45e8-bec4-593ea19a39d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec3411098944d6c9f2dc3a8a2696b8d'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:58.338211 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7b1403b3-916a-45e8-bec4-593ea19a39d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec3411098944d6c9f2dc3a8a2696b8d'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:58.339008 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7b1403b3-916a-45e8-bec4-593ea19a39d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec3411098944d6c9f2dc3a8a2696b8d'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:58.339757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7b1403b3-916a-45e8-bec4-593ea19a39d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec3411098944d6c9f2dc3a8a2696b8d'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:58.361521 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-7b1403b3-916a-45e8-bec4-593ea19a39d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec3411098944d6c9f2dc3a8a2696b8d'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:58.394751 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1207/2400] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:58 2022] DELETE /identity/v3/users/9ec3411098944d6c9f2dc3a8a2696b8d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:53:58.395971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56378365-7e9c-461b-b1bb-e8cfda2f9c4d admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:58.396895 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0126f66-4111-4ddf-a55b-d686aec1fb74 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:58.397188 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0126f66-4111-4ddf-a55b-d686aec1fb74 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:58.397498 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0126f66-4111-4ddf-a55b-d686aec1fb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:58.427140 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-c0126f66-4111-4ddf-a55b-d686aec1fb74 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:53:58.428343 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c0126f66-4111-4ddf-a55b-d686aec1fb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:58.491992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df4e1f-d245-4aaf-8f70-f4ac08580b75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:58.492844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df4e1f-d245-4aaf-8f70-f4ac08580b75 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:58.493223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df4e1f-d245-4aaf-8f70-f4ac08580b75 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:58.493540 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df4e1f-d245-4aaf-8f70-f4ac08580b75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:58.495785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d5df4e1f-d245-4aaf-8f70-f4ac08580b75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:58.539139 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1208/2401] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:53:58.540203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0126f66-4111-4ddf-a55b-d686aec1fb74 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:58.542323 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c0126f66-4111-4ddf-a55b-d686aec1fb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:58.557661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0126f66-4111-4ddf-a55b-d686aec1fb74 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:58.563276 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c6930-9c5e-4ec2-96bc-0d11863fae3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:58.564021 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c6930-9c5e-4ec2-96bc-0d11863fae3c service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:58.564311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c6930-9c5e-4ec2-96bc-0d11863fae3c service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:58.564622 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c6930-9c5e-4ec2-96bc-0d11863fae3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:58.575136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df4e1f-d245-4aaf-8f70-f4ac08580b75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:58.577045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df4e1f-d245-4aaf-8f70-f4ac08580b75 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:58.588560 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1194/2402] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:58.590762 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c6930-9c5e-4ec2-96bc-0d11863fae3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:58.592366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c6930-9c5e-4ec2-96bc-0d11863fae3c service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:58.603360 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1209/2403] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:53:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:53:59.113918 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df4e1f-d245-4aaf-8f70-f4ac08580b75 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:59.116177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d5df4e1f-d245-4aaf-8f70-f4ac08580b75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:59.136047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df4e1f-d245-4aaf-8f70-f4ac08580b75 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:59.148003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:59.148956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:59.149318 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:59.149666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] PATH_INFO: `/v3/projects/2cda036215bb472daf8c7c9050a19fa8` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:59.152288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cda036215bb472daf8c7c9050a19fa8)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:59.155109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:59.155485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c6930-9c5e-4ec2-96bc-0d11863fae3c service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:59.156929 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-103c6930-9c5e-4ec2-96bc-0d11863fae3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:59.180297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c6930-9c5e-4ec2-96bc-0d11863fae3c service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:59.184906 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:59.185853 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:59.186317 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:59.186604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] PATH_INFO: `/v3/projects/1350fbd6055e4a91b6cda001b6fa827a` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:59.193332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1350fbd6055e4a91b6cda001b6fa827a)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:59.193332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:59.244973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cda036215bb472daf8c7c9050a19fa8', 'request_id': 'req-46abe4b3-d7a9-4be7-9fd8-7376d522820d'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.252511 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1350fbd6055e4a91b6cda001b6fa827a', 'request_id': 'req-24e564d1-507f-4b30-b4a6-f7a786b21303'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.257528 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cda036215bb472daf8c7c9050a19fa8', 'request_id': 'req-46abe4b3-d7a9-4be7-9fd8-7376d522820d'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.269754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1350fbd6055e4a91b6cda001b6fa827a', 'request_id': 'req-24e564d1-507f-4b30-b4a6-f7a786b21303'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.273065 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cda036215bb472daf8c7c9050a19fa8', 'request_id': 'req-46abe4b3-d7a9-4be7-9fd8-7376d522820d'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.282489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1350fbd6055e4a91b6cda001b6fa827a', 'request_id': 'req-24e564d1-507f-4b30-b4a6-f7a786b21303'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.294002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] The token cache is being invalidate because project 1350fbd6055e4a91b6cda001b6fa827a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:59.294532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.295210 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] The token cache is being invalidate because project 2cda036215bb472daf8c7c9050a19fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:53:59.295410 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.295535 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.296162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.301312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1210/2404] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:59 2022] DELETE /identity/v3/projects/1350fbd6055e4a91b6cda001b6fa827a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:53:59.301473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1195/2405] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:53:59 2022] DELETE /identity/v3/projects/2cda036215bb472daf8c7c9050a19fa8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:53:59.301570 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:59.302856 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:53:59.303106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:59.304331 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:53:59.399693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abe4b3-d7a9-4be7-9fd8-7376d522820d admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:59.405779 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21c895-c794-44ba-aa5e-1172b72d9d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:59.406849 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21c895-c794-44ba-aa5e-1172b72d9d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:59.407294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21c895-c794-44ba-aa5e-1172b72d9d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:59.407545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21c895-c794-44ba-aa5e-1172b72d9d03 admin admin] PATH_INFO: `/v3/users/462398cf8e6f4ffaaa27da0986e236d4` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:59.408788 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21c895-c794-44ba-aa5e-1172b72d9d03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=462398cf8e6f4ffaaa27da0986e236d4)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:59.410735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21c895-c794-44ba-aa5e-1172b72d9d03 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:59.411676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e564d1-507f-4b30-b4a6-f7a786b21303 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:53:59.418575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd28f2f-2a6c-4b09-a8f4-9a04777d4df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:53:59.419816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd28f2f-2a6c-4b09-a8f4-9a04777d4df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:53:59.420612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd28f2f-2a6c-4b09-a8f4-9a04777d4df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:53:59.421198 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd28f2f-2a6c-4b09-a8f4-9a04777d4df4 admin admin] PATH_INFO: `/v3/users/c00d95e8226e4388b505f28df799bc05` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:53:59.422834 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd28f2f-2a6c-4b09-a8f4-9a04777d4df4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c00d95e8226e4388b505f28df799bc05)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:53:59.425949 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd28f2f-2a6c-4b09-a8f4-9a04777d4df4 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:53:59.478399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ac21c895-c794-44ba-aa5e-1172b72d9d03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '462398cf8e6f4ffaaa27da0986e236d4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.496027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ac21c895-c794-44ba-aa5e-1172b72d9d03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '462398cf8e6f4ffaaa27da0986e236d4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.496872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ac21c895-c794-44ba-aa5e-1172b72d9d03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '462398cf8e6f4ffaaa27da0986e236d4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.497712 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ac21c895-c794-44ba-aa5e-1172b72d9d03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '462398cf8e6f4ffaaa27da0986e236d4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.507604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-4cd28f2f-2a6c-4b09-a8f4-9a04777d4df4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c00d95e8226e4388b505f28df799bc05'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.514978 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ac21c895-c794-44ba-aa5e-1172b72d9d03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '462398cf8e6f4ffaaa27da0986e236d4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.530948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-4cd28f2f-2a6c-4b09-a8f4-9a04777d4df4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c00d95e8226e4388b505f28df799bc05'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.532925 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-4cd28f2f-2a6c-4b09-a8f4-9a04777d4df4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c00d95e8226e4388b505f28df799bc05'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.533657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-4cd28f2f-2a6c-4b09-a8f4-9a04777d4df4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c00d95e8226e4388b505f28df799bc05'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.538001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1196/2406] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:59 2022] DELETE /identity/v3/users/462398cf8e6f4ffaaa27da0986e236d4 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:53:59.559745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-4cd28f2f-2a6c-4b09-a8f4-9a04777d4df4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c00d95e8226e4388b505f28df799bc05'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:53:59.585080 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1211/2407] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:53:59 2022] DELETE /identity/v3/users/c00d95e8226e4388b505f28df799bc05 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:54:00.053249 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21c895-c794-44ba-aa5e-1172b72d9d03 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:00.053728 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ac21c895-c794-44ba-aa5e-1172b72d9d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:00.116009 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd28f2f-2a6c-4b09-a8f4-9a04777d4df4 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:00.117862 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-4cd28f2f-2a6c-4b09-a8f4-9a04777d4df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:00.193614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21c895-c794-44ba-aa5e-1172b72d9d03 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:00.238758 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd28f2f-2a6c-4b09-a8f4-9a04777d4df4 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:00.254695 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b186eb5-7b51-4a2a-add4-1559dda80154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:00.259089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b186eb5-7b51-4a2a-add4-1559dda80154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:00.259376 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b186eb5-7b51-4a2a-add4-1559dda80154 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:00.259657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b186eb5-7b51-4a2a-add4-1559dda80154 admin admin] PATH_INFO: `/v3/projects/72138e97086b4d67bb462e2dfcb067ba` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:00.261130 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b186eb5-7b51-4a2a-add4-1559dda80154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72138e97086b4d67bb462e2dfcb067ba)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:00.266886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b186eb5-7b51-4a2a-add4-1559dda80154 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:00.295614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:00.297215 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:00.297502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:00.297809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] PATH_INFO: `/v3/projects/489500d32bd347b8a49cb3a68befadc4` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:00.299645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=489500d32bd347b8a49cb3a68befadc4)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:00.302040 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:00.353213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0b186eb5-7b51-4a2a-add4-1559dda80154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72138e97086b4d67bb462e2dfcb067ba', 'request_id': 'req-0b186eb5-7b51-4a2a-add4-1559dda80154'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.361603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0b186eb5-7b51-4a2a-add4-1559dda80154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72138e97086b4d67bb462e2dfcb067ba', 'request_id': 'req-0b186eb5-7b51-4a2a-add4-1559dda80154'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.375816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0b186eb5-7b51-4a2a-add4-1559dda80154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72138e97086b4d67bb462e2dfcb067ba', 'request_id': 'req-0b186eb5-7b51-4a2a-add4-1559dda80154'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.393692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '489500d32bd347b8a49cb3a68befadc4', 'request_id': 'req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.396394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0b186eb5-7b51-4a2a-add4-1559dda80154 admin admin] The token cache is being invalidate because project 72138e97086b4d67bb462e2dfcb067ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:54:00.396693 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0b186eb5-7b51-4a2a-add4-1559dda80154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.397359 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0b186eb5-7b51-4a2a-add4-1559dda80154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.399445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1212/2408] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:54:00 2022] DELETE /identity/v3/projects/72138e97086b4d67bb462e2dfcb067ba => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:54:00.404874 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6750ab-d335-4435-9ea0-675d23272e65 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:00.405368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6750ab-d335-4435-9ea0-675d23272e65 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:00.405706 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6750ab-d335-4435-9ea0-675d23272e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:00.406955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '489500d32bd347b8a49cb3a68befadc4', 'request_id': 'req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.425032 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '489500d32bd347b8a49cb3a68befadc4', 'request_id': 'req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.440991 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] The token cache is being invalidate because project 489500d32bd347b8a49cb3a68befadc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:54:00.441281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.442679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.447163 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1197/2409] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:54:00 2022] DELETE /identity/v3/projects/489500d32bd347b8a49cb3a68befadc4 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:00.451113 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:00.452686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:00.471794 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-2a6750ab-d335-4435-9ea0-675d23272e65 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:00.473066 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-2a6750ab-d335-4435-9ea0-675d23272e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:00.553546 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f313c-d0f0-441c-bc8a-c799da2a5f75 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:00.559112 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1213/2410] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:00.560525 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b94566-68a6-4eb8-8ed3-4a551567aaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:00.563169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b94566-68a6-4eb8-8ed3-4a551567aaa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:00.563730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f075-ee21-420c-a0a6-d03434438244 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:00.564071 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f075-ee21-420c-a0a6-d03434438244 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:00.564480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b94566-68a6-4eb8-8ed3-4a551567aaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:00.564761 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b94566-68a6-4eb8-8ed3-4a551567aaa5 admin admin] PATH_INFO: `/v3/users/d4d6e7f065104df384724c70f633c27c` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:00.565193 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f075-ee21-420c-a0a6-d03434438244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:00.567963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b94566-68a6-4eb8-8ed3-4a551567aaa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4d6e7f065104df384724c70f633c27c)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:00.570251 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b94566-68a6-4eb8-8ed3-4a551567aaa5 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:00.584532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-62f7f075-ee21-420c-a0a6-d03434438244 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:00.585906 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-62f7f075-ee21-420c-a0a6-d03434438244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:00.596209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1214/2411] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:54:00.601554 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7f075-ee21-420c-a0a6-d03434438244 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:00.612483 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7f075-ee21-420c-a0a6-d03434438244 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:00.617700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242ab5c-60ad-434f-b135-93780b7b4cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:00.618502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242ab5c-60ad-434f-b135-93780b7b4cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:00.618774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242ab5c-60ad-434f-b135-93780b7b4cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:00.618965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242ab5c-60ad-434f-b135-93780b7b4cc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:00.620189 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242ab5c-60ad-434f-b135-93780b7b4cc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:54:00.620518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242ab5c-60ad-434f-b135-93780b7b4cc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:00.622222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242ab5c-60ad-434f-b135-93780b7b4cc5 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:00.638588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-32b94566-68a6-4eb8-8ed3-4a551567aaa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4d6e7f065104df384724c70f633c27c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.643153 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1215/2412] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:54:00.647061 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242ab5c-60ad-434f-b135-93780b7b4cc5 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:00.658995 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-32b94566-68a6-4eb8-8ed3-4a551567aaa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4d6e7f065104df384724c70f633c27c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.659754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-32b94566-68a6-4eb8-8ed3-4a551567aaa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4d6e7f065104df384724c70f633c27c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.660400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-32b94566-68a6-4eb8-8ed3-4a551567aaa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4d6e7f065104df384724c70f633c27c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.665581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242ab5c-60ad-434f-b135-93780b7b4cc5 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:00.668844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-32b94566-68a6-4eb8-8ed3-4a551567aaa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4d6e7f065104df384724c70f633c27c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:00.671924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ccbe6-6442-47cf-ac2f-47ecbb36b1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:00.672869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ccbe6-6442-47cf-ac2f-47ecbb36b1b8 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:00.673155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ccbe6-6442-47cf-ac2f-47ecbb36b1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:00.673437 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ccbe6-6442-47cf-ac2f-47ecbb36b1b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:00.674334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ccbe6-6442-47cf-ac2f-47ecbb36b1b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:00.677189 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ccbe6-6442-47cf-ac2f-47ecbb36b1b8 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:00.695749 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1198/2413] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:54:00 2022] DELETE /identity/v3/users/d4d6e7f065104df384724c70f633c27c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:54:00.726525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1216/2414] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:54:00 2022] POST /identity/v3/projects => generated 361 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:54:00.730828 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b94566-68a6-4eb8-8ed3-4a551567aaa5 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:00.732895 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-32b94566-68a6-4eb8-8ed3-4a551567aaa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:00.816086 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b94566-68a6-4eb8-8ed3-4a551567aaa5 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:00.882686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9182f9-b424-4ce1-9ab9-66ae847ce498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:00.883212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9182f9-b424-4ce1-9ab9-66ae847ce498 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:00.883574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9182f9-b424-4ce1-9ab9-66ae847ce498 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:00.883926 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9182f9-b424-4ce1-9ab9-66ae847ce498 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:00.884746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9182f9-b424-4ce1-9ab9-66ae847ce498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:00.886781 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9182f9-b424-4ce1-9ab9-66ae847ce498 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:00.919112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1199/2415] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:54:00 2022] POST /identity/v3/users => generated 281 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:54:00.921869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ccbe6-6442-47cf-ac2f-47ecbb36b1b8 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:00.940595 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ccbe6-6442-47cf-ac2f-47ecbb36b1b8 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:00.948238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ccb808-3b7e-4946-a89f-6b804b0bc9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:00.949292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccb808-3b7e-4946-a89f-6b804b0bc9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:00.949622 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccb808-3b7e-4946-a89f-6b804b0bc9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:00.949944 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccb808-3b7e-4946-a89f-6b804b0bc9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:00.950995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ccb808-3b7e-4946-a89f-6b804b0bc9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:00.954584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ccb808-3b7e-4946-a89f-6b804b0bc9a3 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:00.972964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1217/2416] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:54:00.973597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9182f9-b424-4ce1-9ab9-66ae847ce498 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:00.988625 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9182f9-b424-4ce1-9ab9-66ae847ce498 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:01.000217 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fac431-1e27-4857-92c0-b635037833b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:01.000896 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fac431-1e27-4857-92c0-b635037833b1 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.001181 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fac431-1e27-4857-92c0-b635037833b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.001459 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fac431-1e27-4857-92c0-b635037833b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.002187 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fac431-1e27-4857-92c0-b635037833b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:01.005073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fac431-1e27-4857-92c0-b635037833b1 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:01.027809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1200/2417] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:54:01.034196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ccb808-3b7e-4946-a89f-6b804b0bc9a3 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:01.051254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ccb808-3b7e-4946-a89f-6b804b0bc9a3 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:01.058965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d92a63-3641-419a-905b-d284a0bf6c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:01.059961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d92a63-3641-419a-905b-d284a0bf6c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.060411 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d92a63-3641-419a-905b-d284a0bf6c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.061387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d92a63-3641-419a-905b-d284a0bf6c8e admin admin] PATH_INFO: `/v3/projects/514baa8d0fe343a182760afdadfecd5c/users/5091811cde994a5eaae4f3a43aab737a/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.081517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d92a63-3641-419a-905b-d284a0bf6c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=514baa8d0fe343a182760afdadfecd5c, user_id=5091811cde994a5eaae4f3a43aab737a, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:01.088651 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d92a63-3641-419a-905b-d284a0bf6c8e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:01.117326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1218/2418] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:54:01 2022] PUT /identity/v3/projects/514baa8d0fe343a182760afdadfecd5c/users/5091811cde994a5eaae4f3a43aab737a/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:01.118147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdaedd-4b50-4581-a5ea-34406283554a None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.118615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdaedd-4b50-4581-a5ea-34406283554a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.118737 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdaedd-4b50-4581-a5ea-34406283554a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.162480 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d92a63-3641-419a-905b-d284a0bf6c8e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:01.164568 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a7d92a63-3641-419a-905b-d284a0bf6c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:01.173335 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-3afdaedd-4b50-4581-a5ea-34406283554a None None] MFA Rules not processed for user `5091811cde994a5eaae4f3a43aab737a`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:01.176019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-3afdaedd-4b50-4581-a5ea-34406283554a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:01.253834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1201/2419] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:54:01.257441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d92a63-3641-419a-905b-d284a0bf6c8e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:01.266142 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929d848-102f-440b-9cf9-3ca166d35542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:01.267154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929d848-102f-440b-9cf9-3ca166d35542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.267448 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929d848-102f-440b-9cf9-3ca166d35542 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.267730 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929d848-102f-440b-9cf9-3ca166d35542 admin admin] PATH_INFO: `/v3/projects/c2a821f22e63499fb82018a3feb7ff9c` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.269962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929d848-102f-440b-9cf9-3ca166d35542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2a821f22e63499fb82018a3feb7ff9c)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:01.271508 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929d848-102f-440b-9cf9-3ca166d35542 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:01.287566 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdaedd-4b50-4581-a5ea-34406283554a None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:01.289574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-3afdaedd-4b50-4581-a5ea-34406283554a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:01.330255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9929d848-102f-440b-9cf9-3ca166d35542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2a821f22e63499fb82018a3feb7ff9c', 'request_id': 'req-9929d848-102f-440b-9cf9-3ca166d35542'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:01.337320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9929d848-102f-440b-9cf9-3ca166d35542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2a821f22e63499fb82018a3feb7ff9c', 'request_id': 'req-9929d848-102f-440b-9cf9-3ca166d35542'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:01.347279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9929d848-102f-440b-9cf9-3ca166d35542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2a821f22e63499fb82018a3feb7ff9c', 'request_id': 'req-9929d848-102f-440b-9cf9-3ca166d35542'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:01.350933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdaedd-4b50-4581-a5ea-34406283554a None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:01.359208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9929d848-102f-440b-9cf9-3ca166d35542 admin admin] The token cache is being invalidate because project c2a821f22e63499fb82018a3feb7ff9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:54:01.359500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9929d848-102f-440b-9cf9-3ca166d35542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:01.360171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9929d848-102f-440b-9cf9-3ca166d35542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:01.362053 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1219/2420] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:54:01 2022] DELETE /identity/v3/projects/c2a821f22e63499fb82018a3feb7ff9c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:54:01.366915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f4195-db7e-4be8-9798-883079d861e3 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.367128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f4195-db7e-4be8-9798-883079d861e3 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.367390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f4195-db7e-4be8-9798-883079d861e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.387562 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7e8f4195-db7e-4be8-9798-883079d861e3 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:01.389362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7e8f4195-db7e-4be8-9798-883079d861e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:01.415167 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25548c0-0780-4f19-8a44-0a7333ac62cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:01.415825 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25548c0-0780-4f19-8a44-0a7333ac62cb service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.416109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25548c0-0780-4f19-8a44-0a7333ac62cb service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.416393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25548c0-0780-4f19-8a44-0a7333ac62cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.418414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b25548c0-0780-4f19-8a44-0a7333ac62cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:01.454916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1220/2421] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:54:01.461701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd9b33-77b8-4fe3-849f-37a5708b9fe6 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.462019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd9b33-77b8-4fe3-849f-37a5708b9fe6 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.462296 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd9b33-77b8-4fe3-849f-37a5708b9fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.480519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7ecd9b33-77b8-4fe3-849f-37a5708b9fe6 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:01.481701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7ecd9b33-77b8-4fe3-849f-37a5708b9fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:01.483613 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25548c0-0780-4f19-8a44-0a7333ac62cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:01.485164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25548c0-0780-4f19-8a44-0a7333ac62cb service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:01.491094 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1221/2422] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:54:01.495027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1202/2423] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:54:01.495424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd9b33-77b8-4fe3-849f-37a5708b9fe6 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:01.511731 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd9b33-77b8-4fe3-849f-37a5708b9fe6 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:01.511731 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9e7cb-9676-4a39-a46d-c675c2df2c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:01.512128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9e7cb-9676-4a39-a46d-c675c2df2c2c admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.512408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9e7cb-9676-4a39-a46d-c675c2df2c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.512679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9e7cb-9676-4a39-a46d-c675c2df2c2c admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.512958 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25548c0-0780-4f19-8a44-0a7333ac62cb service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:01.513434 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9e7cb-9676-4a39-a46d-c675c2df2c2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:54:01.513775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9e7cb-9676-4a39-a46d-c675c2df2c2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:01.514555 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b25548c0-0780-4f19-8a44-0a7333ac62cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:01.515482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9e7cb-9676-4a39-a46d-c675c2df2c2c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:01.528230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1222/2424] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:54:01.528372 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25548c0-0780-4f19-8a44-0a7333ac62cb service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:01.532482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9e7cb-9676-4a39-a46d-c675c2df2c2c admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:01.533474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a4452-6118-4fad-a011-827ba7a75bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:01.534349 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a4452-6118-4fad-a011-827ba7a75bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.534636 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a4452-6118-4fad-a011-827ba7a75bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.534966 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a4452-6118-4fad-a011-827ba7a75bf1 admin admin] PATH_INFO: `/v3/users/5091811cde994a5eaae4f3a43aab737a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.536339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a4452-6118-4fad-a011-827ba7a75bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5091811cde994a5eaae4f3a43aab737a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:01.538337 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a4452-6118-4fad-a011-827ba7a75bf1 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:01.543770 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9e7cb-9676-4a39-a46d-c675c2df2c2c admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:01.549593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d410f29-fe2d-42dd-a691-4807c67455f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:01.550250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d410f29-fe2d-42dd-a691-4807c67455f1 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.550535 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d410f29-fe2d-42dd-a691-4807c67455f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.550852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d410f29-fe2d-42dd-a691-4807c67455f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.551540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d410f29-fe2d-42dd-a691-4807c67455f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:01.553643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d410f29-fe2d-42dd-a691-4807c67455f1 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:01.587902 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1223/2425] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:54:01 2022] POST /identity/v3/projects => generated 397 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:54:01.591851 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d410f29-fe2d-42dd-a691-4807c67455f1 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:01.596093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e96a4452-6118-4fad-a011-827ba7a75bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5091811cde994a5eaae4f3a43aab737a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:01.608651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e96a4452-6118-4fad-a011-827ba7a75bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5091811cde994a5eaae4f3a43aab737a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:01.609483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e96a4452-6118-4fad-a011-827ba7a75bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5091811cde994a5eaae4f3a43aab737a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:01.610110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e96a4452-6118-4fad-a011-827ba7a75bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5091811cde994a5eaae4f3a43aab737a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:01.617069 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e96a4452-6118-4fad-a011-827ba7a75bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5091811cde994a5eaae4f3a43aab737a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:01.630162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1203/2426] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:54:01 2022] DELETE /identity/v3/users/5091811cde994a5eaae4f3a43aab737a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:54:01.634844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937866a4-6478-4215-81a8-584a77368e0b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.635200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937866a4-6478-4215-81a8-584a77368e0b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.635600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937866a4-6478-4215-81a8-584a77368e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.653725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-937866a4-6478-4215-81a8-584a77368e0b None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:01.654977 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-937866a4-6478-4215-81a8-584a77368e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:01.662357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d410f29-fe2d-42dd-a691-4807c67455f1 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:01.715530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1204/2427] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:01.716665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f6625-eeb2-4efa-bc58-88ca4b24288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:01.717355 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f6625-eeb2-4efa-bc58-88ca4b24288b admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.717635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f6625-eeb2-4efa-bc58-88ca4b24288b admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.717907 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f6625-eeb2-4efa-bc58-88ca4b24288b admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.718583 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f6625-eeb2-4efa-bc58-88ca4b24288b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:01.720541 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f6625-eeb2-4efa-bc58-88ca4b24288b admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:01.723485 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937866a4-6478-4215-81a8-584a77368e0b None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:01.724876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-937866a4-6478-4215-81a8-584a77368e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:01.743761 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1224/2428] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:54:01 2022] POST /identity/v3/users => generated 299 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:54:01.747904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f6625-eeb2-4efa-bc58-88ca4b24288b admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:01.751194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-df0f6625-eeb2-4efa-bc58-88ca4b24288b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:01.765332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f6625-eeb2-4efa-bc58-88ca4b24288b admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:01.770138 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b13ef-4b0d-4a14-ae0c-3462ac9b5161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:01.770979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b13ef-4b0d-4a14-ae0c-3462ac9b5161 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.771264 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b13ef-4b0d-4a14-ae0c-3462ac9b5161 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.771540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b13ef-4b0d-4a14-ae0c-3462ac9b5161 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.772266 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b13ef-4b0d-4a14-ae0c-3462ac9b5161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:01.773898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b13ef-4b0d-4a14-ae0c-3462ac9b5161 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:01.784654 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1225/2429] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:54:01.788502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b13ef-4b0d-4a14-ae0c-3462ac9b5161 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:01.791627 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937866a4-6478-4215-81a8-584a77368e0b None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:01.796117 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e40cf6-131d-4360-9166-c43ec3817bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:01.796843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e40cf6-131d-4360-9166-c43ec3817bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.797361 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e40cf6-131d-4360-9166-c43ec3817bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.797709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e40cf6-131d-4360-9166-c43ec3817bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.799249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b13ef-4b0d-4a14-ae0c-3462ac9b5161 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:01.804010 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eca7c8-9e98-4f45-a1fd-2645ba223e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:01.804664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eca7c8-9e98-4f45-a1fd-2645ba223e40 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.804950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eca7c8-9e98-4f45-a1fd-2645ba223e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.805233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eca7c8-9e98-4f45-a1fd-2645ba223e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.806096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eca7c8-9e98-4f45-a1fd-2645ba223e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:01.807757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eca7c8-9e98-4f45-a1fd-2645ba223e40 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:01.811446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e40cf6-131d-4360-9166-c43ec3817bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:01.812899 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e40cf6-131d-4360-9166-c43ec3817bc5 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:01.823196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1226/2430] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:54:01.823821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1205/2431] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:54:01.824089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eca7c8-9e98-4f45-a1fd-2645ba223e40 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:01.834184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eca7c8-9e98-4f45-a1fd-2645ba223e40 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:01.838865 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea329d5-0bd2-4455-a6e5-34891a0eab74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:01.839733 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea329d5-0bd2-4455-a6e5-34891a0eab74 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.840023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea329d5-0bd2-4455-a6e5-34891a0eab74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.840307 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea329d5-0bd2-4455-a6e5-34891a0eab74 admin admin] PATH_INFO: `/v3/projects/34e659abaf6142d788fc71f19eff63b4/users/b42da9410d894b15852f3eb7b35576bd/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.855119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea329d5-0bd2-4455-a6e5-34891a0eab74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34e659abaf6142d788fc71f19eff63b4, user_id=b42da9410d894b15852f3eb7b35576bd, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:01.858591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea329d5-0bd2-4455-a6e5-34891a0eab74 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:01.872410 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1227/2432] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:54:01 2022] PUT /identity/v3/projects/34e659abaf6142d788fc71f19eff63b4/users/b42da9410d894b15852f3eb7b35576bd/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:54:01.883900 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c54f3-e2d8-4393-8081-e07de5c747c7 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:01.884283 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c54f3-e2d8-4393-8081-e07de5c747c7 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:01.884629 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c54f3-e2d8-4393-8081-e07de5c747c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:01.911518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-0a9c54f3-e2d8-4393-8081-e07de5c747c7 None None] MFA Rules not processed for user `b42da9410d894b15852f3eb7b35576bd`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:01.912740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0a9c54f3-e2d8-4393-8081-e07de5c747c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:01.981812 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1206/2433] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:01.990622 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea329d5-0bd2-4455-a6e5-34891a0eab74 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:01.992432 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-cea329d5-0bd2-4455-a6e5-34891a0eab74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:02.063865 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea329d5-0bd2-4455-a6e5-34891a0eab74 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:02.067601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5f60c-0c83-47af-a2f7-e8baa5bee295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:02.068252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5f60c-0c83-47af-a2f7-e8baa5bee295 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:02.068534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5f60c-0c83-47af-a2f7-e8baa5bee295 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:02.068812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5f60c-0c83-47af-a2f7-e8baa5bee295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:02.077433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5f60c-0c83-47af-a2f7-e8baa5bee295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:02.079105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5f60c-0c83-47af-a2f7-e8baa5bee295 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:02.093105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1228/2434] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:54:02.230053 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c54f3-e2d8-4393-8081-e07de5c747c7 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:02.231465 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0a9c54f3-e2d8-4393-8081-e07de5c747c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:02.298226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c54f3-e2d8-4393-8081-e07de5c747c7 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:02.302733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2718f48-3977-4218-9b13-db428240e87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:02.303627 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2718f48-3977-4218-9b13-db428240e87a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:02.303929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2718f48-3977-4218-9b13-db428240e87a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:02.304209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2718f48-3977-4218-9b13-db428240e87a admin admin] PATH_INFO: `/v3/projects/514baa8d0fe343a182760afdadfecd5c` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:02.305420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2718f48-3977-4218-9b13-db428240e87a admin admin] RBAC: Authorizing `identity:delete_project(project_id=514baa8d0fe343a182760afdadfecd5c)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:02.307367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2718f48-3977-4218-9b13-db428240e87a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:02.353031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e2718f48-3977-4218-9b13-db428240e87a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '514baa8d0fe343a182760afdadfecd5c', 'request_id': 'req-e2718f48-3977-4218-9b13-db428240e87a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:02.359463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e2718f48-3977-4218-9b13-db428240e87a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '514baa8d0fe343a182760afdadfecd5c', 'request_id': 'req-e2718f48-3977-4218-9b13-db428240e87a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:02.367318 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e2718f48-3977-4218-9b13-db428240e87a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '514baa8d0fe343a182760afdadfecd5c', 'request_id': 'req-e2718f48-3977-4218-9b13-db428240e87a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:02.378697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e2718f48-3977-4218-9b13-db428240e87a admin admin] The token cache is being invalidate because project 514baa8d0fe343a182760afdadfecd5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:54:02.379017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e2718f48-3977-4218-9b13-db428240e87a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:02.379727 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e2718f48-3977-4218-9b13-db428240e87a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:02.381752 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1207/2435] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:54:02 2022] DELETE /identity/v3/projects/514baa8d0fe343a182760afdadfecd5c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:02.387754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17c6ed-f84b-4db6-8caa-4593cc31b96d None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:02.388273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17c6ed-f84b-4db6-8caa-4593cc31b96d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:02.388372 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17c6ed-f84b-4db6-8caa-4593cc31b96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:02.407582 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-6a17c6ed-f84b-4db6-8caa-4593cc31b96d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:02.410207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-6a17c6ed-f84b-4db6-8caa-4593cc31b96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:02.490286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1229/2436] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:02.497820 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b176c-d3a5-4e1f-8461-e72e1fb40624 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:02.498257 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b176c-d3a5-4e1f-8461-e72e1fb40624 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:02.498648 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b176c-d3a5-4e1f-8461-e72e1fb40624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:02.519210 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-0b9b176c-d3a5-4e1f-8461-e72e1fb40624 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:02.520427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0b9b176c-d3a5-4e1f-8461-e72e1fb40624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:02.534593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1208/2437] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:02.535966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17c6ed-f84b-4db6-8caa-4593cc31b96d None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:02.545656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17c6ed-f84b-4db6-8caa-4593cc31b96d None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:02.550482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43568c9-f8b6-4872-8543-ba282dc50854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:02.551063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43568c9-f8b6-4872-8543-ba282dc50854 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:02.551350 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43568c9-f8b6-4872-8543-ba282dc50854 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:02.551627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43568c9-f8b6-4872-8543-ba282dc50854 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:02.552376 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43568c9-f8b6-4872-8543-ba282dc50854 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:54:02.552686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43568c9-f8b6-4872-8543-ba282dc50854 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:02.554297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43568c9-f8b6-4872-8543-ba282dc50854 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:02.566749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1230/2438] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:54:02.570796 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b176c-d3a5-4e1f-8461-e72e1fb40624 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:02.582080 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b176c-d3a5-4e1f-8461-e72e1fb40624 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:02.587134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cd4bc-f43c-4314-99f6-7a2315f4c896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:02.587784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cd4bc-f43c-4314-99f6-7a2315f4c896 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:02.588097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cd4bc-f43c-4314-99f6-7a2315f4c896 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:02.588375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cd4bc-f43c-4314-99f6-7a2315f4c896 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:02.589060 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599cd4bc-f43c-4314-99f6-7a2315f4c896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:02.591079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599cd4bc-f43c-4314-99f6-7a2315f4c896 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:02.618900 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1209/2439] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:54:02 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:54:02.621986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43568c9-f8b6-4872-8543-ba282dc50854 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:02.632561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43568c9-f8b6-4872-8543-ba282dc50854 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:02.692322 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e764b3-1404-4600-b9d8-aff46ef24bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:02.692981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e764b3-1404-4600-b9d8-aff46ef24bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:02.693266 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e764b3-1404-4600-b9d8-aff46ef24bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:02.693536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e764b3-1404-4600-b9d8-aff46ef24bb0 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:02.694208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e764b3-1404-4600-b9d8-aff46ef24bb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:02.696170 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e764b3-1404-4600-b9d8-aff46ef24bb0 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:02.715224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1231/2440] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:54:02 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:54:02.719475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cd4bc-f43c-4314-99f6-7a2315f4c896 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:02.729575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cd4bc-f43c-4314-99f6-7a2315f4c896 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:02.734192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064de213-90d9-4192-9724-3063932577f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:02.734885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064de213-90d9-4192-9724-3063932577f9 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:02.735168 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064de213-90d9-4192-9724-3063932577f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:02.735446 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064de213-90d9-4192-9724-3063932577f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:02.736200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064de213-90d9-4192-9724-3063932577f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:02.737788 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064de213-90d9-4192-9724-3063932577f9 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:02.748594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1210/2441] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:54:02.752248 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e764b3-1404-4600-b9d8-aff46ef24bb0 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:02.762233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e764b3-1404-4600-b9d8-aff46ef24bb0 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:02.767887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbfd18-743c-4928-a76a-c30db8335f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:02.768541 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbfd18-743c-4928-a76a-c30db8335f17 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:02.768827 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbfd18-743c-4928-a76a-c30db8335f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:02.769104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbfd18-743c-4928-a76a-c30db8335f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:02.769817 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbfd18-743c-4928-a76a-c30db8335f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:02.771461 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbfd18-743c-4928-a76a-c30db8335f17 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:02.781173 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1232/2442] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:54:02.785164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064de213-90d9-4192-9724-3063932577f9 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:02.794947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064de213-90d9-4192-9724-3063932577f9 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:02.799531 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025dc8e9-2bc6-40c5-b8d9-035eb6ba6fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:02.800393 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dc8e9-2bc6-40c5-b8d9-035eb6ba6fad admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:02.800674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dc8e9-2bc6-40c5-b8d9-035eb6ba6fad admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:02.800946 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dc8e9-2bc6-40c5-b8d9-035eb6ba6fad admin admin] PATH_INFO: `/v3/projects/7129585a2e644c3bbea78b22f4ee9786/users/23c2854279ae4774a226feaede3c3de5/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:02.815373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025dc8e9-2bc6-40c5-b8d9-035eb6ba6fad admin admin] RBAC: Authorizing `identity:create_grant(project_id=7129585a2e644c3bbea78b22f4ee9786, user_id=23c2854279ae4774a226feaede3c3de5, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:02.818027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025dc8e9-2bc6-40c5-b8d9-035eb6ba6fad admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:02.831690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1211/2443] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:54:02 2022] PUT /identity/v3/projects/7129585a2e644c3bbea78b22f4ee9786/users/23c2854279ae4774a226feaede3c3de5/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:02.841600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f4542-eac3-4a5a-83b0-5ae332f1c84b None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:02.841947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f4542-eac3-4a5a-83b0-5ae332f1c84b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:02.842256 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f4542-eac3-4a5a-83b0-5ae332f1c84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:02.868270 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7c1f4542-eac3-4a5a-83b0-5ae332f1c84b None None] MFA Rules not processed for user `23c2854279ae4774a226feaede3c3de5`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:02.869385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7c1f4542-eac3-4a5a-83b0-5ae332f1c84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:02.930105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1233/2444] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:02.941098 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025dc8e9-2bc6-40c5-b8d9-035eb6ba6fad admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:02.942943 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-025dc8e9-2bc6-40c5-b8d9-035eb6ba6fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:03.005564 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025dc8e9-2bc6-40c5-b8d9-035eb6ba6fad admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:03.009205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0adaff-f249-4823-b888-6dfbc75a344a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:03.009851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0adaff-f249-4823-b888-6dfbc75a344a service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:03.010194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0adaff-f249-4823-b888-6dfbc75a344a service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:03.010478 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0adaff-f249-4823-b888-6dfbc75a344a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:03.019280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0adaff-f249-4823-b888-6dfbc75a344a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:03.020716 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0adaff-f249-4823-b888-6dfbc75a344a service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:03.029422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1212/2445] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:54:08.468966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a2897-6f86-42e9-a87a-38dff7df7067 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:08.468966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a2897-6f86-42e9-a87a-38dff7df7067 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:08.468966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a2897-6f86-42e9-a87a-38dff7df7067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:08.506820 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e51a2897-6f86-42e9-a87a-38dff7df7067 None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:08.507423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e51a2897-6f86-42e9-a87a-38dff7df7067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:08.594139 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1234/2446] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:54:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:08.613306 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0adaff-f249-4823-b888-6dfbc75a344a service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:08.615309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0c0adaff-f249-4823-b888-6dfbc75a344a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:08.686742 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc12e7-c027-4b6d-b2db-824dd72d9034 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:08.687425 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc12e7-c027-4b6d-b2db-824dd72d9034 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:08.688020 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc12e7-c027-4b6d-b2db-824dd72d9034 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:08.689671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1235/2447] 10.222.0.35 () {54 vars in 921 bytes} [Fri May 13 21:54:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 21:54:08.696807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911778c-68a8-45ad-a216-d86fd1d0c97b None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:08.696807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911778c-68a8-45ad-a216-d86fd1d0c97b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:08.696992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911778c-68a8-45ad-a216-d86fd1d0c97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:08.738376 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-3911778c-68a8-45ad-a216-d86fd1d0c97b None None] MFA Rules not processed for user `316ef43c6c304bd3a27e085bf4da8a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:08.739918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3911778c-68a8-45ad-a216-d86fd1d0c97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:08.776670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0adaff-f249-4823-b888-6dfbc75a344a service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:08.792968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48730f7-7a56-4abc-99cc-919e1b577bf7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:08.793859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48730f7-7a56-4abc-99cc-919e1b577bf7 service swift] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:08.794015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48730f7-7a56-4abc-99cc-919e1b577bf7 service swift] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:08.794193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48730f7-7a56-4abc-99cc-919e1b577bf7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:08.813420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48730f7-7a56-4abc-99cc-919e1b577bf7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:08.815708 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48730f7-7a56-4abc-99cc-919e1b577bf7 service swift] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:08.831818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1213/2448] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:54:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:54:08.890030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1236/2449] 10.222.0.35 () {58 vars in 1018 bytes} [Fri May 13 21:54:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:54:08.913431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48730f7-7a56-4abc-99cc-919e1b577bf7 service swift] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:08.915451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f48730f7-7a56-4abc-99cc-919e1b577bf7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:09.058669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48730f7-7a56-4abc-99cc-919e1b577bf7 service swift] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:09.078391 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c87719-ad63-40ea-ac31-63ad8cb223fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad57b046f3a43978d1c98bcc1a76d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:09.079142 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c87719-ad63-40ea-ac31-63ad8cb223fc service glance] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:09.079622 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c87719-ad63-40ea-ac31-63ad8cb223fc service glance] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:09.079916 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c87719-ad63-40ea-ac31-63ad8cb223fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:09.100965 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c87719-ad63-40ea-ac31-63ad8cb223fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:09.111660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c87719-ad63-40ea-ac31-63ad8cb223fc service glance] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:09.146087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1214/2450] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:54:09.217672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911778c-68a8-45ad-a216-d86fd1d0c97b None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:09.219532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3911778c-68a8-45ad-a216-d86fd1d0c97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:09.354927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6d7a5-d3b8-42e2-8f84-4ae2589e9bb0 None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:09.355603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6d7a5-d3b8-42e2-8f84-4ae2589e9bb0 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:09.356196 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6d7a5-d3b8-42e2-8f84-4ae2589e9bb0 None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:09.358306 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1215/2451] 10.222.0.35 () {54 vars in 921 bytes} [Fri May 13 21:54:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:54:09.367274 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a6de0-a062-47b0-bc4d-6f09730f7531 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:09.368127 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a6de0-a062-47b0-bc4d-6f09730f7531 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:09.368708 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a6de0-a062-47b0-bc4d-6f09730f7531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:09.411590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-b10a6de0-a062-47b0-bc4d-6f09730f7531 None None] MFA Rules not processed for user `c4df058606c448a1bb757020e869ac33`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:09.416995 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b10a6de0-a062-47b0-bc4d-6f09730f7531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:09.434919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911778c-68a8-45ad-a216-d86fd1d0c97b None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:09.485353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9165a-f60b-4fd0-b632-039259ae348c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:09.489473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9165a-f60b-4fd0-b632-039259ae348c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:09.490093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9165a-f60b-4fd0-b632-039259ae348c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:09.490592 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9165a-f60b-4fd0-b632-039259ae348c admin admin] PATH_INFO: `/v3/users/f2ea289c76224c67a43267e86bc723a3` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:09.493099 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9165a-f60b-4fd0-b632-039259ae348c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2ea289c76224c67a43267e86bc723a3)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:09.496200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9165a-f60b-4fd0-b632-039259ae348c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:09.578191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1216/2452] 10.222.0.35 () {58 vars in 1018 bytes} [Fri May 13 21:54:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:09.631268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-c6b9165a-f60b-4fd0-b632-039259ae348c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2ea289c76224c67a43267e86bc723a3'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:09.661798 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a6de0-a062-47b0-bc4d-6f09730f7531 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:09.663520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b10a6de0-a062-47b0-bc4d-6f09730f7531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:09.667903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-c6b9165a-f60b-4fd0-b632-039259ae348c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2ea289c76224c67a43267e86bc723a3'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:09.668747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-c6b9165a-f60b-4fd0-b632-039259ae348c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2ea289c76224c67a43267e86bc723a3'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:09.670601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-c6b9165a-f60b-4fd0-b632-039259ae348c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2ea289c76224c67a43267e86bc723a3'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:09.679104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a6de0-a062-47b0-bc4d-6f09730f7531 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:09.698874 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195b6f7-9e10-44d7-9755-fbe02cc60016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:09.699208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-c6b9165a-f60b-4fd0-b632-039259ae348c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2ea289c76224c67a43267e86bc723a3'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:09.699604 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195b6f7-9e10-44d7-9755-fbe02cc60016 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:09.699898 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195b6f7-9e10-44d7-9755-fbe02cc60016 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:09.700184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195b6f7-9e10-44d7-9755-fbe02cc60016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:09.709197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b195b6f7-9e10-44d7-9755-fbe02cc60016 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:09.722937 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1237/2453] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:54:09 2022] DELETE /identity/v3/users/f2ea289c76224c67a43267e86bc723a3 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:09.722937 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9165a-f60b-4fd0-b632-039259ae348c admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:09.724113 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c6b9165a-f60b-4fd0-b632-039259ae348c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:09.743676 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195b6f7-9e10-44d7-9755-fbe02cc60016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:09.745286 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195b6f7-9e10-44d7-9755-fbe02cc60016 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:09.768651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1217/2454] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:54:09.770290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b607e-4aae-460f-8042-df7c5a826620 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:09.773818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b607e-4aae-460f-8042-df7c5a826620 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:09.774288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b607e-4aae-460f-8042-df7c5a826620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:09.807385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ce5b607e-4aae-460f-8042-df7c5a826620 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:09.811984 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ce5b607e-4aae-460f-8042-df7c5a826620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:09.893096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9165a-f60b-4fd0-b632-039259ae348c admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:09.904737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7466f-1c24-4537-9892-ae93204f75ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:09.905427 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7466f-1c24-4537-9892-ae93204f75ad service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:09.905707 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7466f-1c24-4537-9892-ae93204f75ad service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:09.905979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7466f-1c24-4537-9892-ae93204f75ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:09.910758 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-eec7466f-1c24-4537-9892-ae93204f75ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:09.974223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1218/2455] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:10.015659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b607e-4aae-460f-8042-df7c5a826620 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:10.017072 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ce5b607e-4aae-460f-8042-df7c5a826620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:10.045716 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b607e-4aae-460f-8042-df7c5a826620 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:10.063400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc35333-faa1-4787-b6a3-bab0620b75be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:10.064072 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35333-faa1-4787-b6a3-bab0620b75be service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:10.064358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35333-faa1-4787-b6a3-bab0620b75be service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:10.064680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35333-faa1-4787-b6a3-bab0620b75be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:10.080611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc35333-faa1-4787-b6a3-bab0620b75be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:10.082075 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc35333-faa1-4787-b6a3-bab0620b75be service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:10.087128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7466f-1c24-4537-9892-ae93204f75ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:10.089968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7466f-1c24-4537-9892-ae93204f75ad service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:10.109343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1219/2456] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:54:10.132609 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1238/2457] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:54:10.509919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751f27b-5944-47d3-8bd2-c2dd0a71311f None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:10.510207 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751f27b-5944-47d3-8bd2-c2dd0a71311f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:10.510486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751f27b-5944-47d3-8bd2-c2dd0a71311f None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:10.512166 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1220/2458] 10.222.0.35 () {54 vars in 921 bytes} [Fri May 13 21:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:54:10.519440 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf2db6-367d-4f1d-9611-4fa14fd2d550 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:10.519949 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf2db6-367d-4f1d-9611-4fa14fd2d550 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:10.519949 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf2db6-367d-4f1d-9611-4fa14fd2d550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:10.540395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-05bf2db6-367d-4f1d-9611-4fa14fd2d550 None None] MFA Rules not processed for user `316ef43c6c304bd3a27e085bf4da8a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:10.541541 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-05bf2db6-367d-4f1d-9611-4fa14fd2d550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:10.610408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1239/2459] 10.222.0.35 () {58 vars in 1018 bytes} [Fri May 13 21:54:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:10.621088 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751f27b-5944-47d3-8bd2-c2dd0a71311f None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:10.632864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751f27b-5944-47d3-8bd2-c2dd0a71311f None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:10.636985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c8ea4-3eb3-4c30-a06d-35269011a357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:10.637740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c8ea4-3eb3-4c30-a06d-35269011a357 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:10.638033 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c8ea4-3eb3-4c30-a06d-35269011a357 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:10.638472 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c8ea4-3eb3-4c30-a06d-35269011a357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:10.649171 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c8ea4-3eb3-4c30-a06d-35269011a357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:10.650633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c8ea4-3eb3-4c30-a06d-35269011a357 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:10.662652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1221/2460] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:54:10.913244 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf2db6-367d-4f1d-9611-4fa14fd2d550 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:10.914940 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-05bf2db6-367d-4f1d-9611-4fa14fd2d550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:10.929491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf2db6-367d-4f1d-9611-4fa14fd2d550 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:10.934393 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca392713-d554-4a6b-a04d-0c2329a97c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:10.935589 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca392713-d554-4a6b-a04d-0c2329a97c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:10.936170 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca392713-d554-4a6b-a04d-0c2329a97c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:10.936520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca392713-d554-4a6b-a04d-0c2329a97c62 admin admin] PATH_INFO: `/v3/projects/7476c4e1892840df92f9a9dc382e85f9` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:10.937848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca392713-d554-4a6b-a04d-0c2329a97c62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7476c4e1892840df92f9a9dc382e85f9)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:10.940060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca392713-d554-4a6b-a04d-0c2329a97c62 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:10.997720 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ca392713-d554-4a6b-a04d-0c2329a97c62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7476c4e1892840df92f9a9dc382e85f9', 'request_id': 'req-ca392713-d554-4a6b-a04d-0c2329a97c62'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:11.006741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ca392713-d554-4a6b-a04d-0c2329a97c62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7476c4e1892840df92f9a9dc382e85f9', 'request_id': 'req-ca392713-d554-4a6b-a04d-0c2329a97c62'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:11.015475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ca392713-d554-4a6b-a04d-0c2329a97c62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7476c4e1892840df92f9a9dc382e85f9', 'request_id': 'req-ca392713-d554-4a6b-a04d-0c2329a97c62'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:11.026620 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ca392713-d554-4a6b-a04d-0c2329a97c62 admin admin] The token cache is being invalidate because project 7476c4e1892840df92f9a9dc382e85f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:54:11.026955 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ca392713-d554-4a6b-a04d-0c2329a97c62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:11.027602 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-ca392713-d554-4a6b-a04d-0c2329a97c62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:11.029688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1240/2461] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:54:10 2022] DELETE /identity/v3/projects/7476c4e1892840df92f9a9dc382e85f9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:11.034832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3fa0b-0d62-40e0-b28f-215d71791283 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:11.035294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3fa0b-0d62-40e0-b28f-215d71791283 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:11.035757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3fa0b-0d62-40e0-b28f-215d71791283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:11.054906 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-faf3fa0b-0d62-40e0-b28f-215d71791283 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:11.056043 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-faf3fa0b-0d62-40e0-b28f-215d71791283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:11.115601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1222/2462] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:54:11.123387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbcaa2-1389-4613-a235-05b9168a2553 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:11.123790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbcaa2-1389-4613-a235-05b9168a2553 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:11.124472 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbcaa2-1389-4613-a235-05b9168a2553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:11.142362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-9ccbcaa2-1389-4613-a235-05b9168a2553 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:11.143708 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9ccbcaa2-1389-4613-a235-05b9168a2553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:11.158538 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1241/2463] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:11.163290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3fa0b-0d62-40e0-b28f-215d71791283 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:11.174135 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3fa0b-0d62-40e0-b28f-215d71791283 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:11.178866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f479e6-87ea-43b3-9b73-78d6313fa68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:11.179550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f479e6-87ea-43b3-9b73-78d6313fa68e admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:11.179830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f479e6-87ea-43b3-9b73-78d6313fa68e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:11.180106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f479e6-87ea-43b3-9b73-78d6313fa68e admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:11.180852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f479e6-87ea-43b3-9b73-78d6313fa68e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:54:11.181154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f479e6-87ea-43b3-9b73-78d6313fa68e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:11.182785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f479e6-87ea-43b3-9b73-78d6313fa68e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:11.198637 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1223/2464] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:54:11.205370 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbcaa2-1389-4613-a235-05b9168a2553 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:11.214535 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbcaa2-1389-4613-a235-05b9168a2553 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:11.219947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643a712-88e6-4d07-8c79-079186fa2178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:11.220732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643a712-88e6-4d07-8c79-079186fa2178 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:11.221022 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643a712-88e6-4d07-8c79-079186fa2178 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:11.221299 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643a712-88e6-4d07-8c79-079186fa2178 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:11.221982 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643a712-88e6-4d07-8c79-079186fa2178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:11.223975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643a712-88e6-4d07-8c79-079186fa2178 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:11.251928 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1242/2465] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:54:11 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:54:11.253826 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f479e6-87ea-43b3-9b73-78d6313fa68e admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:11.265062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f479e6-87ea-43b3-9b73-78d6313fa68e admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:11.304214 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ddd3b-02de-4319-9ab8-2de19fe7d4ef None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:11.304318 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ddd3b-02de-4319-9ab8-2de19fe7d4ef None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:11.304513 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ddd3b-02de-4319-9ab8-2de19fe7d4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:11.332254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-ec2ddd3b-02de-4319-9ab8-2de19fe7d4ef None None] MFA Rules not processed for user `6b11a3de8b71482594a04ab7e53a94bd`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:11.334218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225fac-4624-4cff-bfda-105de5d6c557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:11.335170 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225fac-4624-4cff-bfda-105de5d6c557 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:11.335990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225fac-4624-4cff-bfda-105de5d6c557 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:11.336552 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225fac-4624-4cff-bfda-105de5d6c557 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:11.337708 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf225fac-4624-4cff-bfda-105de5d6c557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:11.340890 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf225fac-4624-4cff-bfda-105de5d6c557 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:11.344510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ec2ddd3b-02de-4319-9ab8-2de19fe7d4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:11.376460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1224/2466] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:54:11 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:54:11.380492 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225fac-4624-4cff-bfda-105de5d6c557 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:11.396984 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225fac-4624-4cff-bfda-105de5d6c557 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:11.402348 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83199674-fae6-4563-b6e2-fa46c7ec6f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:11.403185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83199674-fae6-4563-b6e2-fa46c7ec6f0f admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:11.403468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83199674-fae6-4563-b6e2-fa46c7ec6f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:11.403747 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83199674-fae6-4563-b6e2-fa46c7ec6f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:11.405042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83199674-fae6-4563-b6e2-fa46c7ec6f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:11.406667 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83199674-fae6-4563-b6e2-fa46c7ec6f0f admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:11.422553 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1225/2467] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:54:11.423703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83199674-fae6-4563-b6e2-fa46c7ec6f0f admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:11.435545 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1243/2468] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:11.435760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83199674-fae6-4563-b6e2-fa46c7ec6f0f admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:11.443111 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824ec49-b791-416a-a44a-275626c1cfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:11.443893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824ec49-b791-416a-a44a-275626c1cfeb admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:11.444309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824ec49-b791-416a-a44a-275626c1cfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:11.444670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824ec49-b791-416a-a44a-275626c1cfeb admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:11.445525 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824ec49-b791-416a-a44a-275626c1cfeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:11.447443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824ec49-b791-416a-a44a-275626c1cfeb admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:11.463919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1226/2469] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:54:11.468088 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ddd3b-02de-4319-9ab8-2de19fe7d4ef None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:11.484583 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ddd3b-02de-4319-9ab8-2de19fe7d4ef None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:11.491416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de472c-c491-46e5-8666-44f6ec6c597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:11.492610 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de472c-c491-46e5-8666-44f6ec6c597a admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:11.493307 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de472c-c491-46e5-8666-44f6ec6c597a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:11.493737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de472c-c491-46e5-8666-44f6ec6c597a admin admin] PATH_INFO: `/v3/projects/d37b24e8a85f49ce9d043e8a38b8748f/users/4812f8677a754dcd9e7a5607c643e8cb/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:11.519678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de472c-c491-46e5-8666-44f6ec6c597a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d37b24e8a85f49ce9d043e8a38b8748f, user_id=4812f8677a754dcd9e7a5607c643e8cb, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:11.523381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de472c-c491-46e5-8666-44f6ec6c597a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:11.541251 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1244/2470] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:54:11 2022] PUT /identity/v3/projects/d37b24e8a85f49ce9d043e8a38b8748f/users/4812f8677a754dcd9e7a5607c643e8cb/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:11.546387 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a0d11-2d1a-4b0f-87d0-b43fc0c658d5 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:11.546683 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a0d11-2d1a-4b0f-87d0-b43fc0c658d5 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:11.547002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a0d11-2d1a-4b0f-87d0-b43fc0c658d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:11.565012 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ab2a0d11-2d1a-4b0f-87d0-b43fc0c658d5 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:11.566381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ab2a0d11-2d1a-4b0f-87d0-b43fc0c658d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:11.648277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1227/2471] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:11.659980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de472c-c491-46e5-8666-44f6ec6c597a admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:11.661697 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d3de472c-c491-46e5-8666-44f6ec6c597a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:11.734325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de472c-c491-46e5-8666-44f6ec6c597a admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:11.738185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc5c87-0d1f-49c6-a7bb-cde2b77f658c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:11.739464 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc5c87-0d1f-49c6-a7bb-cde2b77f658c service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:11.739850 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc5c87-0d1f-49c6-a7bb-cde2b77f658c service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:11.740223 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc5c87-0d1f-49c6-a7bb-cde2b77f658c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:11.750488 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc5c87-0d1f-49c6-a7bb-cde2b77f658c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:11.752095 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc5c87-0d1f-49c6-a7bb-cde2b77f658c service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:11.762300 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1245/2472] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:54:13.495852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a0d11-2d1a-4b0f-87d0-b43fc0c658d5 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:13.498689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ab2a0d11-2d1a-4b0f-87d0-b43fc0c658d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:13.513606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a0d11-2d1a-4b0f-87d0-b43fc0c658d5 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:13.526178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2622a7-ea80-479b-ba9e-dded68570b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:13.527100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2622a7-ea80-479b-ba9e-dded68570b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:13.527422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2622a7-ea80-479b-ba9e-dded68570b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:13.527669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2622a7-ea80-479b-ba9e-dded68570b0a admin admin] PATH_INFO: `/v3/users/23c2854279ae4774a226feaede3c3de5` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:13.529978 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2622a7-ea80-479b-ba9e-dded68570b0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c2854279ae4774a226feaede3c3de5)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:13.556763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2622a7-ea80-479b-ba9e-dded68570b0a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:13.752463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0a2622a7-ea80-479b-ba9e-dded68570b0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c2854279ae4774a226feaede3c3de5'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:13.855150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0a2622a7-ea80-479b-ba9e-dded68570b0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c2854279ae4774a226feaede3c3de5'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:13.859550 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0a2622a7-ea80-479b-ba9e-dded68570b0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c2854279ae4774a226feaede3c3de5'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:13.861517 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0a2622a7-ea80-479b-ba9e-dded68570b0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c2854279ae4774a226feaede3c3de5'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:13.883155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0a2622a7-ea80-479b-ba9e-dded68570b0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c2854279ae4774a226feaede3c3de5'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:13.917124 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1228/2473] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:54:13 2022] DELETE /identity/v3/users/23c2854279ae4774a226feaede3c3de5 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:13.922107 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba98b84-a748-4bc5-b22f-200ffe0b0e97 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:13.922598 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba98b84-a748-4bc5-b22f-200ffe0b0e97 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:13.922819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba98b84-a748-4bc5-b22f-200ffe0b0e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:13.973333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-dba98b84-a748-4bc5-b22f-200ffe0b0e97 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:13.975122 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-dba98b84-a748-4bc5-b22f-200ffe0b0e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:14.262435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1246/2474] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:14.296881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2622a7-ea80-479b-ba9e-dded68570b0a admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:14.298391 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0a2622a7-ea80-479b-ba9e-dded68570b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:14.511261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2622a7-ea80-479b-ba9e-dded68570b0a admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:14.535841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e674e7-46ba-4992-b303-df8e8249f77e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:14.542061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e674e7-46ba-4992-b303-df8e8249f77e service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:14.544190 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e674e7-46ba-4992-b303-df8e8249f77e service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:14.548339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e674e7-46ba-4992-b303-df8e8249f77e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:14.594636 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e674e7-46ba-4992-b303-df8e8249f77e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:14.598530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e674e7-46ba-4992-b303-df8e8249f77e service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:14.624619 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1229/2475] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:54:16.301886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba98b84-a748-4bc5-b22f-200ffe0b0e97 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:16.304087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-dba98b84-a748-4bc5-b22f-200ffe0b0e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:16.324122 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba98b84-a748-4bc5-b22f-200ffe0b0e97 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:16.339676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:16.340875 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:16.341379 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:16.341859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] PATH_INFO: `/v3/projects/7129585a2e644c3bbea78b22f4ee9786` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:16.345732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=7129585a2e644c3bbea78b22f4ee9786)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:16.348585 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:16.554872 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7129585a2e644c3bbea78b22f4ee9786', 'request_id': 'req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:16.572951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7129585a2e644c3bbea78b22f4ee9786', 'request_id': 'req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:16.593922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7129585a2e644c3bbea78b22f4ee9786', 'request_id': 'req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:16.609546 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] The token cache is being invalidate because project 7129585a2e644c3bbea78b22f4ee9786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:54:16.609848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:16.612419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:16.619304 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1247/2476] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:54:16 2022] DELETE /identity/v3/projects/7129585a2e644c3bbea78b22f4ee9786 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:54:16.620652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366577-5de0-4d4f-9562-db570b595ecb None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:16.621462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366577-5de0-4d4f-9562-db570b595ecb None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:16.622170 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366577-5de0-4d4f-9562-db570b595ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:16.645462 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:16.646717 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:16.709147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-59366577-5de0-4d4f-9562-db570b595ecb None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:16.710400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-59366577-5de0-4d4f-9562-db570b595ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:16.897821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114ae05-86b6-4dc1-8f11-6931f4dfa9fa admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:16.908166 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a80992-03ae-4437-be06-2b85ac2a4cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:16.913342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a80992-03ae-4437-be06-2b85ac2a4cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:16.913674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a80992-03ae-4437-be06-2b85ac2a4cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:16.913958 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a80992-03ae-4437-be06-2b85ac2a4cf2 admin admin] PATH_INFO: `/v3/users/b42da9410d894b15852f3eb7b35576bd` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:16.929513 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a80992-03ae-4437-be06-2b85ac2a4cf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b42da9410d894b15852f3eb7b35576bd)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:16.947023 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a80992-03ae-4437-be06-2b85ac2a4cf2 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:16.987692 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1230/2477] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:16.989965 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fe51a-5ed6-4b53-a64f-f7a71e28b320 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:16.990454 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fe51a-5ed6-4b53-a64f-f7a71e28b320 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:16.991761 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fe51a-5ed6-4b53-a64f-f7a71e28b320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:17.074907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-a29fe51a-5ed6-4b53-a64f-f7a71e28b320 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:17.080615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a29fe51a-5ed6-4b53-a64f-f7a71e28b320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:17.112558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1231/2478] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:17.118204 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fe51a-5ed6-4b53-a64f-f7a71e28b320 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:17.151309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fe51a-5ed6-4b53-a64f-f7a71e28b320 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:17.167470 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-19a80992-03ae-4437-be06-2b85ac2a4cf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b42da9410d894b15852f3eb7b35576bd'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:17.171855 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe3eb5-ae36-4eb9-b8f5-e80d1da3d697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:17.173089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe3eb5-ae36-4eb9-b8f5-e80d1da3d697 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:17.176966 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe3eb5-ae36-4eb9-b8f5-e80d1da3d697 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:17.177593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe3eb5-ae36-4eb9-b8f5-e80d1da3d697 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:17.182968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe3eb5-ae36-4eb9-b8f5-e80d1da3d697 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:54:17.186939 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe3eb5-ae36-4eb9-b8f5-e80d1da3d697 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:17.189814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe3eb5-ae36-4eb9-b8f5-e80d1da3d697 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:17.215676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-19a80992-03ae-4437-be06-2b85ac2a4cf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b42da9410d894b15852f3eb7b35576bd'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:17.217018 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-19a80992-03ae-4437-be06-2b85ac2a4cf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b42da9410d894b15852f3eb7b35576bd'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:17.217686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-19a80992-03ae-4437-be06-2b85ac2a4cf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b42da9410d894b15852f3eb7b35576bd'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:17.240359 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-19a80992-03ae-4437-be06-2b85ac2a4cf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b42da9410d894b15852f3eb7b35576bd'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:17.245829 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1232/2479] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:54:17.257752 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe3eb5-ae36-4eb9-b8f5-e80d1da3d697 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:17.264261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-cafe3eb5-ae36-4eb9-b8f5-e80d1da3d697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:17.271879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1248/2480] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:54:16 2022] DELETE /identity/v3/users/b42da9410d894b15852f3eb7b35576bd => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:17.276071 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbd8cd-3dd0-436c-aad6-9f4a2c320b1e None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:17.276460 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbd8cd-3dd0-436c-aad6-9f4a2c320b1e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:17.276749 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbd8cd-3dd0-436c-aad6-9f4a2c320b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:17.315827 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-06bbd8cd-3dd0-436c-aad6-9f4a2c320b1e None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:17.317065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-06bbd8cd-3dd0-436c-aad6-9f4a2c320b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:17.338215 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe3eb5-ae36-4eb9-b8f5-e80d1da3d697 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:17.455681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1249/2481] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:54:17.472864 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbd8cd-3dd0-436c-aad6-9f4a2c320b1e None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:17.474330 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-06bbd8cd-3dd0-436c-aad6-9f4a2c320b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:17.519493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ed2d3-f1d3-41d9-ac05-b943d8ac60f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:17.520517 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ed2d3-f1d3-41d9-ac05-b943d8ac60f3 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:17.520517 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ed2d3-f1d3-41d9-ac05-b943d8ac60f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:17.520724 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ed2d3-f1d3-41d9-ac05-b943d8ac60f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:17.521431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ed2d3-f1d3-41d9-ac05-b943d8ac60f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:17.523400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ed2d3-f1d3-41d9-ac05-b943d8ac60f3 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:17.587873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1233/2482] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:54:17 2022] POST /identity/v3/projects => generated 373 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:54:17.603319 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ed2d3-f1d3-41d9-ac05-b943d8ac60f3 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:17.632048 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbd8cd-3dd0-436c-aad6-9f4a2c320b1e None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:17.722846 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ed2d3-f1d3-41d9-ac05-b943d8ac60f3 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:17.740010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9bd38-bdba-40ca-91af-c02955b147d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:17.740668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9bd38-bdba-40ca-91af-c02955b147d0 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:17.740954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9bd38-bdba-40ca-91af-c02955b147d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:17.741237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9bd38-bdba-40ca-91af-c02955b147d0 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:17.741938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9bd38-bdba-40ca-91af-c02955b147d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:17.744144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9bd38-bdba-40ca-91af-c02955b147d0 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:17.780774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b46298-080b-4b40-8967-a0ba67be661c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:17.785846 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b46298-080b-4b40-8967-a0ba67be661c service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:17.786695 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b46298-080b-4b40-8967-a0ba67be661c service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:17.789536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b46298-080b-4b40-8967-a0ba67be661c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:17.796162 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1234/2483] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:54:17 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:54:17.801919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9bd38-bdba-40ca-91af-c02955b147d0 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:17.827988 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9bd38-bdba-40ca-91af-c02955b147d0 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:17.830945 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b46298-080b-4b40-8967-a0ba67be661c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:17.836469 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b46298-080b-4b40-8967-a0ba67be661c service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:17.846381 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc336e-b064-48e3-ac81-5ee29ff22357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:17.847268 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc336e-b064-48e3-ac81-5ee29ff22357 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:17.849384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc336e-b064-48e3-ac81-5ee29ff22357 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:17.851454 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc336e-b064-48e3-ac81-5ee29ff22357 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:17.852345 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc336e-b064-48e3-ac81-5ee29ff22357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:17.857578 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc336e-b064-48e3-ac81-5ee29ff22357 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:17.874070 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1250/2484] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:54:17.902136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1235/2485] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:54:17.903796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b46298-080b-4b40-8967-a0ba67be661c service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:17.916780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b46298-080b-4b40-8967-a0ba67be661c service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:17.927982 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7aa363-65ae-46d2-a3da-b31a98ee75a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:17.928686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7aa363-65ae-46d2-a3da-b31a98ee75a2 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:17.928933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7aa363-65ae-46d2-a3da-b31a98ee75a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:17.929210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7aa363-65ae-46d2-a3da-b31a98ee75a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:17.929929 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7aa363-65ae-46d2-a3da-b31a98ee75a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:17.934284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7aa363-65ae-46d2-a3da-b31a98ee75a2 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:17.967547 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1251/2486] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:54:17.971500 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc336e-b064-48e3-ac81-5ee29ff22357 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:18.063739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc336e-b064-48e3-ac81-5ee29ff22357 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:18.081919 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df250bf3-2ef0-44eb-980b-9b3d7cd1b253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:18.085172 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df250bf3-2ef0-44eb-980b-9b3d7cd1b253 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:18.087420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df250bf3-2ef0-44eb-980b-9b3d7cd1b253 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:18.089652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df250bf3-2ef0-44eb-980b-9b3d7cd1b253 admin admin] PATH_INFO: `/v3/projects/a480dd328dbc461690b7a80095a21967/users/75a06208154149a88ca10d30520a58d5/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:18.149626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df250bf3-2ef0-44eb-980b-9b3d7cd1b253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a480dd328dbc461690b7a80095a21967, user_id=75a06208154149a88ca10d30520a58d5, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:18.153497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df250bf3-2ef0-44eb-980b-9b3d7cd1b253 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:18.194486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1236/2487] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:54:17 2022] PUT /identity/v3/projects/a480dd328dbc461690b7a80095a21967/users/75a06208154149a88ca10d30520a58d5/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:18.204809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274bad2-483d-4cc7-b030-e67225be7f54 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:18.205097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274bad2-483d-4cc7-b030-e67225be7f54 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:18.205373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274bad2-483d-4cc7-b030-e67225be7f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:18.350491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-9274bad2-483d-4cc7-b030-e67225be7f54 None None] MFA Rules not processed for user `75a06208154149a88ca10d30520a58d5`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:18.353220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9274bad2-483d-4cc7-b030-e67225be7f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:18.547199 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df250bf3-2ef0-44eb-980b-9b3d7cd1b253 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:18.549242 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-df250bf3-2ef0-44eb-980b-9b3d7cd1b253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:18.564845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1252/2488] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:18.575137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274bad2-483d-4cc7-b030-e67225be7f54 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:18.583174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9274bad2-483d-4cc7-b030-e67225be7f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:18.754061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df250bf3-2ef0-44eb-980b-9b3d7cd1b253 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:18.777768 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274bad2-483d-4cc7-b030-e67225be7f54 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:18.778443 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faee8c9-e5cb-4193-9997-3765f78a594e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:18.781112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faee8c9-e5cb-4193-9997-3765f78a594e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:18.783184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faee8c9-e5cb-4193-9997-3765f78a594e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:18.783631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faee8c9-e5cb-4193-9997-3765f78a594e admin admin] PATH_INFO: `/v3/projects/34e659abaf6142d788fc71f19eff63b4` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:18.789655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faee8c9-e5cb-4193-9997-3765f78a594e admin admin] RBAC: Authorizing `identity:delete_project(project_id=34e659abaf6142d788fc71f19eff63b4)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:18.796540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8aceb-d9e9-436e-b032-5693d75f47ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:18.798971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8aceb-d9e9-436e-b032-5693d75f47ce service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:18.798971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8aceb-d9e9-436e-b032-5693d75f47ce service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:18.798971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8aceb-d9e9-436e-b032-5693d75f47ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:18.799414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faee8c9-e5cb-4193-9997-3765f78a594e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:18.819624 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8aceb-d9e9-436e-b032-5693d75f47ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:18.823472 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8aceb-d9e9-436e-b032-5693d75f47ce service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:18.848961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1253/2489] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:54:19.009253 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9faee8c9-e5cb-4193-9997-3765f78a594e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34e659abaf6142d788fc71f19eff63b4', 'request_id': 'req-9faee8c9-e5cb-4193-9997-3765f78a594e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:19.023484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9faee8c9-e5cb-4193-9997-3765f78a594e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34e659abaf6142d788fc71f19eff63b4', 'request_id': 'req-9faee8c9-e5cb-4193-9997-3765f78a594e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:19.047271 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9faee8c9-e5cb-4193-9997-3765f78a594e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34e659abaf6142d788fc71f19eff63b4', 'request_id': 'req-9faee8c9-e5cb-4193-9997-3765f78a594e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:19.063045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9faee8c9-e5cb-4193-9997-3765f78a594e admin admin] The token cache is being invalidate because project 34e659abaf6142d788fc71f19eff63b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:54:19.063431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9faee8c9-e5cb-4193-9997-3765f78a594e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:19.064068 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9faee8c9-e5cb-4193-9997-3765f78a594e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:19.113174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1237/2490] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:54:18 2022] DELETE /identity/v3/projects/34e659abaf6142d788fc71f19eff63b4 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:20.657403 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa047aa-2124-4113-8687-9a254dc28deb None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:20.658441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa047aa-2124-4113-8687-9a254dc28deb None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:20.659015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa047aa-2124-4113-8687-9a254dc28deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:20.696460 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-3fa047aa-2124-4113-8687-9a254dc28deb None None] MFA Rules not processed for user `75a06208154149a88ca10d30520a58d5`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:20.699580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3fa047aa-2124-4113-8687-9a254dc28deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:20.890375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1254/2491] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:23.117134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed1a39-17d8-4b5e-85d5-3e3dd74c49e9 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:23.117700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed1a39-17d8-4b5e-85d5-3e3dd74c49e9 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:23.132235 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed1a39-17d8-4b5e-85d5-3e3dd74c49e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:23.167248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-f7ed1a39-17d8-4b5e-85d5-3e3dd74c49e9 None None] MFA Rules not processed for user `4812f8677a754dcd9e7a5607c643e8cb`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:23.168868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f7ed1a39-17d8-4b5e-85d5-3e3dd74c49e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:23.279743 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1238/2492] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:23.284660 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa047aa-2124-4113-8687-9a254dc28deb None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:23.286242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3fa047aa-2124-4113-8687-9a254dc28deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:23.395901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa047aa-2124-4113-8687-9a254dc28deb None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:23.404973 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907abfd-9f28-47bf-9e2b-0cb3d7ab0e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:23.406176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907abfd-9f28-47bf-9e2b-0cb3d7ab0e52 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:23.406639 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907abfd-9f28-47bf-9e2b-0cb3d7ab0e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:23.407101 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907abfd-9f28-47bf-9e2b-0cb3d7ab0e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:23.408417 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907abfd-9f28-47bf-9e2b-0cb3d7ab0e52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:23.421248 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907abfd-9f28-47bf-9e2b-0cb3d7ab0e52 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:23.462974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed1a39-17d8-4b5e-85d5-3e3dd74c49e9 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:23.468590 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1255/2493] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:54:23 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:54:23.474613 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed1a39-17d8-4b5e-85d5-3e3dd74c49e9 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:23.574652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d75af0-a66d-43fb-93c4-623cbac78d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:23.575483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d75af0-a66d-43fb-93c4-623cbac78d77 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:23.575771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d75af0-a66d-43fb-93c4-623cbac78d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:23.576064 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d75af0-a66d-43fb-93c4-623cbac78d77 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:23.576765 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d75af0-a66d-43fb-93c4-623cbac78d77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:23.578784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d75af0-a66d-43fb-93c4-623cbac78d77 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:23.605077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1239/2494] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:54:23 2022] POST /identity/v3/users => generated 299 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:54:23.606349 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907abfd-9f28-47bf-9e2b-0cb3d7ab0e52 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:23.620015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907abfd-9f28-47bf-9e2b-0cb3d7ab0e52 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:23.635882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a6a62-5255-4b38-8696-d993b2cec134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:23.636719 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a6a62-5255-4b38-8696-d993b2cec134 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:23.637238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a6a62-5255-4b38-8696-d993b2cec134 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:23.637617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a6a62-5255-4b38-8696-d993b2cec134 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:23.639165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a6a62-5255-4b38-8696-d993b2cec134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:23.641095 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a6a62-5255-4b38-8696-d993b2cec134 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:23.674166 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1256/2495] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:54:23.678899 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d75af0-a66d-43fb-93c4-623cbac78d77 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:23.701618 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d75af0-a66d-43fb-93c4-623cbac78d77 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:23.708014 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63231a4-5266-4e71-848a-577ad4307fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:23.709145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63231a4-5266-4e71-848a-577ad4307fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:23.709499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63231a4-5266-4e71-848a-577ad4307fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:23.709845 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63231a4-5266-4e71-848a-577ad4307fd5 admin admin] PATH_INFO: `/v3/projects/370055efb92b4c5ea20501db02775ba8/users/df5b3cac540e4e6a88d131e757f36a4d/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:23.734857 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63231a4-5266-4e71-848a-577ad4307fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=370055efb92b4c5ea20501db02775ba8, user_id=df5b3cac540e4e6a88d131e757f36a4d, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:23.739751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63231a4-5266-4e71-848a-577ad4307fd5 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:23.774020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1240/2496] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:54:23 2022] PUT /identity/v3/projects/370055efb92b4c5ea20501db02775ba8/users/df5b3cac540e4e6a88d131e757f36a4d/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:23.775279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a6a62-5255-4b38-8696-d993b2cec134 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:23.790966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a6a62-5255-4b38-8696-d993b2cec134 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:23.888797 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caca114-152a-4cfe-a03a-e56753a8672c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:23.889672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caca114-152a-4cfe-a03a-e56753a8672c admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:23.890372 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caca114-152a-4cfe-a03a-e56753a8672c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:23.890746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caca114-152a-4cfe-a03a-e56753a8672c admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:23.891685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caca114-152a-4cfe-a03a-e56753a8672c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:23.893989 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caca114-152a-4cfe-a03a-e56753a8672c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:23.917007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1257/2497] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:54:23.918203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63231a4-5266-4e71-848a-577ad4307fd5 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:23.948789 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63231a4-5266-4e71-848a-577ad4307fd5 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:23.958464 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72539e2-204e-4786-9652-2971772a8d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:23.960096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72539e2-204e-4786-9652-2971772a8d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:23.960892 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72539e2-204e-4786-9652-2971772a8d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:23.961341 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72539e2-204e-4786-9652-2971772a8d18 admin admin] PATH_INFO: `/v3/projects/370055efb92b4c5ea20501db02775ba8/users/df5b3cac540e4e6a88d131e757f36a4d/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:23.965199 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72539e2-204e-4786-9652-2971772a8d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=370055efb92b4c5ea20501db02775ba8, user_id=df5b3cac540e4e6a88d131e757f36a4d, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:23.971927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72539e2-204e-4786-9652-2971772a8d18 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:23.987408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1241/2498] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:54:23 2022] PUT /identity/v3/projects/370055efb92b4c5ea20501db02775ba8/users/df5b3cac540e4e6a88d131e757f36a4d/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:32.157190 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae24833-7ca4-4322-9472-5570b645d367 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:32.157681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae24833-7ca4-4322-9472-5570b645d367 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:32.157681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae24833-7ca4-4322-9472-5570b645d367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:32.205852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-dae24833-7ca4-4322-9472-5570b645d367 None None] MFA Rules not processed for user `df5b3cac540e4e6a88d131e757f36a4d`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:32.207546 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-dae24833-7ca4-4322-9472-5570b645d367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:32.295451 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1258/2499] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:32.622463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72539e2-204e-4786-9652-2971772a8d18 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:32.625141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c72539e2-204e-4786-9652-2971772a8d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:32.732255 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72539e2-204e-4786-9652-2971772a8d18 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:32.736603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c611a48-2dbb-49df-a180-de762697a0e3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad57b046f3a43978d1c98bcc1a76d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:32.737571 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c611a48-2dbb-49df-a180-de762697a0e3 service glance] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:32.738112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c611a48-2dbb-49df-a180-de762697a0e3 service glance] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:32.738922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c611a48-2dbb-49df-a180-de762697a0e3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:32.810852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c611a48-2dbb-49df-a180-de762697a0e3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:32.812682 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c611a48-2dbb-49df-a180-de762697a0e3 service glance] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:32.838060 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1242/2500] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:54:33.105487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f27e3b-d138-4a5b-8d13-89f6238d2f95 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:33.105920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f27e3b-d138-4a5b-8d13-89f6238d2f95 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:33.106395 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f27e3b-d138-4a5b-8d13-89f6238d2f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:33.157559 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-18f27e3b-d138-4a5b-8d13-89f6238d2f95 None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:33.158595 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-18f27e3b-d138-4a5b-8d13-89f6238d2f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:33.359191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1259/2501] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:54:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:33.382744 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c611a48-2dbb-49df-a180-de762697a0e3 service glance] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:33.388588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0c611a48-2dbb-49df-a180-de762697a0e3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:33.557802 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c611a48-2dbb-49df-a180-de762697a0e3 service glance] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:33.571458 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6780a2b-deaa-48fc-b5f3-61c612ace013 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:33.572164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6780a2b-deaa-48fc-b5f3-61c612ace013 service swift] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:33.572431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6780a2b-deaa-48fc-b5f3-61c612ace013 service swift] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:33.572707 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6780a2b-deaa-48fc-b5f3-61c612ace013 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:33.596475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6780a2b-deaa-48fc-b5f3-61c612ace013 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:33.598779 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6780a2b-deaa-48fc-b5f3-61c612ace013 service swift] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:33.627138 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1243/2502] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:54:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:54:42.570976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925ac7d-c0ee-4294-9690-f00c8f2fa27f None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:42.571806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925ac7d-c0ee-4294-9690-f00c8f2fa27f None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:42.571806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925ac7d-c0ee-4294-9690-f00c8f2fa27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:42.635599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-3925ac7d-c0ee-4294-9690-f00c8f2fa27f None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:42.636849 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3925ac7d-c0ee-4294-9690-f00c8f2fa27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:42.655393 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1260/2503] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:54:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:42.678141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6780a2b-deaa-48fc-b5f3-61c612ace013 service swift] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:42.701680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6780a2b-deaa-48fc-b5f3-61c612ace013 service swift] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:42.706927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2aadc8-5af1-457b-b8d3-44ab0e796c21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:42.707588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2aadc8-5af1-457b-b8d3-44ab0e796c21 service swift] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:42.707876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2aadc8-5af1-457b-b8d3-44ab0e796c21 service swift] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:42.708158 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2aadc8-5af1-457b-b8d3-44ab0e796c21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:42.717288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2aadc8-5af1-457b-b8d3-44ab0e796c21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:42.719105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2aadc8-5af1-457b-b8d3-44ab0e796c21 service swift] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:42.728195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1244/2504] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:54:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:54:47.212081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ceaf31-b5e6-432f-93cc-2bab56841d16 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:47.212716 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ceaf31-b5e6-432f-93cc-2bab56841d16 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:47.212716 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ceaf31-b5e6-432f-93cc-2bab56841d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:47.229571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-55ceaf31-b5e6-432f-93cc-2bab56841d16 None None] MFA Rules not processed for user `4812f8677a754dcd9e7a5607c643e8cb`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:47.230796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-55ceaf31-b5e6-432f-93cc-2bab56841d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:47.238111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1261/2505] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:47.981214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8291e5-7ca0-4dde-ae17-2de6858b0e69 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:47.981214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8291e5-7ca0-4dde-ae17-2de6858b0e69 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:47.981214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8291e5-7ca0-4dde-ae17-2de6858b0e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:48.000846 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-4f8291e5-7ca0-4dde-ae17-2de6858b0e69 None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:48.005333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4f8291e5-7ca0-4dde-ae17-2de6858b0e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:48.033384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1245/2506] 10.222.0.35 () {58 vars in 967 bytes} [Fri May 13 21:54:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:48.055239 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ceaf31-b5e6-432f-93cc-2bab56841d16 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:48.071782 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ceaf31-b5e6-432f-93cc-2bab56841d16 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:48.078444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3bb51-9d57-45b3-99d0-f459c0debf62 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:48.079591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3bb51-9d57-45b3-99d0-f459c0debf62 service swift] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:48.080092 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3bb51-9d57-45b3-99d0-f459c0debf62 service swift] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:48.080924 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3bb51-9d57-45b3-99d0-f459c0debf62 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:48.091507 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3bb51-9d57-45b3-99d0-f459c0debf62 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:48.093163 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3bb51-9d57-45b3-99d0-f459c0debf62 service swift] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:48.103119 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1262/2507] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:54:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 13 21:54:50.420640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8291e5-7ca0-4dde-ae17-2de6858b0e69 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:50.422178 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4f8291e5-7ca0-4dde-ae17-2de6858b0e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:50.530251 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8291e5-7ca0-4dde-ae17-2de6858b0e69 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:50.534877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2746279-ff8b-4440-9b58-7ad3dc570008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:50.535830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2746279-ff8b-4440-9b58-7ad3dc570008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:50.536023 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2746279-ff8b-4440-9b58-7ad3dc570008 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:50.536303 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2746279-ff8b-4440-9b58-7ad3dc570008 admin admin] PATH_INFO: `/v3/users/004034ab6aca41ae893214bf0e81841b` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:50.537618 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2746279-ff8b-4440-9b58-7ad3dc570008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=004034ab6aca41ae893214bf0e81841b)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:50.539659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2746279-ff8b-4440-9b58-7ad3dc570008 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:50.603012 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c2746279-ff8b-4440-9b58-7ad3dc570008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '004034ab6aca41ae893214bf0e81841b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:50.616563 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c2746279-ff8b-4440-9b58-7ad3dc570008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '004034ab6aca41ae893214bf0e81841b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:50.617762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c2746279-ff8b-4440-9b58-7ad3dc570008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '004034ab6aca41ae893214bf0e81841b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:50.618364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c2746279-ff8b-4440-9b58-7ad3dc570008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '004034ab6aca41ae893214bf0e81841b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:50.627122 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c2746279-ff8b-4440-9b58-7ad3dc570008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '004034ab6aca41ae893214bf0e81841b'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:50.642079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1246/2508] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:54:50 2022] DELETE /identity/v3/users/004034ab6aca41ae893214bf0e81841b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:50.860544 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3bb51-9d57-45b3-99d0-f459c0debf62 service swift] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:50.878976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3bb51-9d57-45b3-99d0-f459c0debf62 service swift] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:50.943365 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca497b-7451-4b2f-b04c-4108f332ab50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:50.944224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca497b-7451-4b2f-b04c-4108f332ab50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:50.944501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca497b-7451-4b2f-b04c-4108f332ab50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:50.944997 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca497b-7451-4b2f-b04c-4108f332ab50 admin admin] PATH_INFO: `/v3/projects/1aef84932f4f4725bbc4095f07ebf56e` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:50.946250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca497b-7451-4b2f-b04c-4108f332ab50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aef84932f4f4725bbc4095f07ebf56e)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:50.948386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca497b-7451-4b2f-b04c-4108f332ab50 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:50.994673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-deca497b-7451-4b2f-b04c-4108f332ab50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aef84932f4f4725bbc4095f07ebf56e', 'request_id': 'req-deca497b-7451-4b2f-b04c-4108f332ab50'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:51.001324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-deca497b-7451-4b2f-b04c-4108f332ab50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aef84932f4f4725bbc4095f07ebf56e', 'request_id': 'req-deca497b-7451-4b2f-b04c-4108f332ab50'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:51.011275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-deca497b-7451-4b2f-b04c-4108f332ab50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aef84932f4f4725bbc4095f07ebf56e', 'request_id': 'req-deca497b-7451-4b2f-b04c-4108f332ab50'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:51.027466 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-deca497b-7451-4b2f-b04c-4108f332ab50 admin admin] The token cache is being invalidate because project 1aef84932f4f4725bbc4095f07ebf56e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:54:51.027756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-deca497b-7451-4b2f-b04c-4108f332ab50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:51.028962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-deca497b-7451-4b2f-b04c-4108f332ab50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:51.030540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1263/2509] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:54:50 2022] DELETE /identity/v3/projects/1aef84932f4f4725bbc4095f07ebf56e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:51.035394 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c896a-6cdf-4bf3-8969-519d59a7d139 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:51.035607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c896a-6cdf-4bf3-8969-519d59a7d139 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:51.035974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c896a-6cdf-4bf3-8969-519d59a7d139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:51.053699 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-193c896a-6cdf-4bf3-8969-519d59a7d139 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:51.054948 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-193c896a-6cdf-4bf3-8969-519d59a7d139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:51.118155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1247/2510] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:51.126473 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b5ef7-3dc8-4056-92d7-7205b8e6ca75 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:51.126919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b5ef7-3dc8-4056-92d7-7205b8e6ca75 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:51.127300 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b5ef7-3dc8-4056-92d7-7205b8e6ca75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:51.148471 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-708b5ef7-3dc8-4056-92d7-7205b8e6ca75 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:51.150809 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-708b5ef7-3dc8-4056-92d7-7205b8e6ca75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:51.164391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1264/2511] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:51.168365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c896a-6cdf-4bf3-8969-519d59a7d139 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:51.181232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c896a-6cdf-4bf3-8969-519d59a7d139 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:51.186386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d539a6-2059-45c1-96ae-aec6bf207e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:51.187086 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d539a6-2059-45c1-96ae-aec6bf207e27 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:51.187371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d539a6-2059-45c1-96ae-aec6bf207e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:51.187644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d539a6-2059-45c1-96ae-aec6bf207e27 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:51.188536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d539a6-2059-45c1-96ae-aec6bf207e27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:54:51.188999 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d539a6-2059-45c1-96ae-aec6bf207e27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:51.190620 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d539a6-2059-45c1-96ae-aec6bf207e27 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:51.203905 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1248/2512] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:54:51.208391 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b5ef7-3dc8-4056-92d7-7205b8e6ca75 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:51.225848 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b5ef7-3dc8-4056-92d7-7205b8e6ca75 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:51.231106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92e7d6-0ac7-4527-bc25-ea26aeca9e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:51.232403 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92e7d6-0ac7-4527-bc25-ea26aeca9e5e admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:51.232678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92e7d6-0ac7-4527-bc25-ea26aeca9e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:51.232949 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92e7d6-0ac7-4527-bc25-ea26aeca9e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:51.233641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92e7d6-0ac7-4527-bc25-ea26aeca9e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:51.235796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92e7d6-0ac7-4527-bc25-ea26aeca9e5e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:51.273106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1265/2513] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:54:51 2022] POST /identity/v3/projects => generated 369 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:54:51.277611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d539a6-2059-45c1-96ae-aec6bf207e27 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:51.309897 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d539a6-2059-45c1-96ae-aec6bf207e27 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:51.398971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fc6fb-6d09-4c41-a254-c87db6e913eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:51.399655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fc6fb-6d09-4c41-a254-c87db6e913eb admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:51.400022 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fc6fb-6d09-4c41-a254-c87db6e913eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:51.400301 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fc6fb-6d09-4c41-a254-c87db6e913eb admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:51.401010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1fc6fb-6d09-4c41-a254-c87db6e913eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:51.406575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1fc6fb-6d09-4c41-a254-c87db6e913eb admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:51.447468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1249/2514] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:54:51 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:54:51.452388 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92e7d6-0ac7-4527-bc25-ea26aeca9e5e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:51.463394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92e7d6-0ac7-4527-bc25-ea26aeca9e5e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:51.468933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c6f44-b486-47d4-8c88-d0214bf7c109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:51.469594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c6f44-b486-47d4-8c88-d0214bf7c109 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:51.469881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c6f44-b486-47d4-8c88-d0214bf7c109 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:51.470154 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c6f44-b486-47d4-8c88-d0214bf7c109 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:51.470920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c6f44-b486-47d4-8c88-d0214bf7c109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:51.472521 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c6f44-b486-47d4-8c88-d0214bf7c109 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:51.483741 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1266/2515] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:54:51.487860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fc6fb-6d09-4c41-a254-c87db6e913eb admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:51.498286 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fc6fb-6d09-4c41-a254-c87db6e913eb admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:51.503351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475ac7f-979a-43b0-a5fe-a0cd9f88aa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:51.504007 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475ac7f-979a-43b0-a5fe-a0cd9f88aa88 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:51.504283 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475ac7f-979a-43b0-a5fe-a0cd9f88aa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:51.504556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475ac7f-979a-43b0-a5fe-a0cd9f88aa88 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:51.505296 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475ac7f-979a-43b0-a5fe-a0cd9f88aa88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:51.506913 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475ac7f-979a-43b0-a5fe-a0cd9f88aa88 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:51.517100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1250/2516] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:54:51.522728 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c6f44-b486-47d4-8c88-d0214bf7c109 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:51.534384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c6f44-b486-47d4-8c88-d0214bf7c109 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:51.539688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ca30f-eebe-48e6-b6f5-64f978622bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:51.540584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ca30f-eebe-48e6-b6f5-64f978622bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:51.540918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ca30f-eebe-48e6-b6f5-64f978622bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:51.541241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ca30f-eebe-48e6-b6f5-64f978622bcf admin admin] PATH_INFO: `/v3/projects/afd714bddd13492cb9bfb9eb20c40e98/users/838b4334782b418f9ac593d04fda373f/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:51.558915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ca30f-eebe-48e6-b6f5-64f978622bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd714bddd13492cb9bfb9eb20c40e98, user_id=838b4334782b418f9ac593d04fda373f, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:51.562238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ca30f-eebe-48e6-b6f5-64f978622bcf admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:51.595487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1267/2517] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:54:51 2022] PUT /identity/v3/projects/afd714bddd13492cb9bfb9eb20c40e98/users/838b4334782b418f9ac593d04fda373f/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:51.599388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1ae40-17b1-47d4-b34a-ecd1f60653b0 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:51.599923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1ae40-17b1-47d4-b34a-ecd1f60653b0 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:51.600225 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1ae40-17b1-47d4-b34a-ecd1f60653b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:51.647472 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-dfb1ae40-17b1-47d4-b34a-ecd1f60653b0 None None] MFA Rules not processed for user `838b4334782b418f9ac593d04fda373f`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:51.649290 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-dfb1ae40-17b1-47d4-b34a-ecd1f60653b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:51.724256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1251/2518] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:51.737486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ca30f-eebe-48e6-b6f5-64f978622bcf admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:51.738672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c47ca30f-eebe-48e6-b6f5-64f978622bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:51.819325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ca30f-eebe-48e6-b6f5-64f978622bcf admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:51.823021 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22058e-3c56-427f-bb8d-70644c4ec410 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad57b046f3a43978d1c98bcc1a76d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:51.823731 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22058e-3c56-427f-bb8d-70644c4ec410 service glance] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:51.824015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22058e-3c56-427f-bb8d-70644c4ec410 service glance] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:51.824290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22058e-3c56-427f-bb8d-70644c4ec410 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:51.834685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22058e-3c56-427f-bb8d-70644c4ec410 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:51.836677 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22058e-3c56-427f-bb8d-70644c4ec410 service glance] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:51.851995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1268/2519] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:51 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:54:54.237862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1ae40-17b1-47d4-b34a-ecd1f60653b0 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:54.239596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-dfb1ae40-17b1-47d4-b34a-ecd1f60653b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:54.377702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1ae40-17b1-47d4-b34a-ecd1f60653b0 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:54.383407 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7d0fa-fff9-44ab-ac9e-1716c3a5d949 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8124ab8abcc640c5a48b3e0216afa48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:54.384199 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7d0fa-fff9-44ab-ac9e-1716c3a5d949 service placement] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:54.384336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7d0fa-fff9-44ab-ac9e-1716c3a5d949 service placement] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:54.385176 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7d0fa-fff9-44ab-ac9e-1716c3a5d949 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:54.386712 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-dfc7d0fa-fff9-44ab-ac9e-1716c3a5d949 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:54.399194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7d0fa-fff9-44ab-ac9e-1716c3a5d949 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:54.400814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7d0fa-fff9-44ab-ac9e-1716c3a5d949 service placement] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:54.426752 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1252/2520] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:54:56.768184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475525f2-2526-4da6-b00a-9c0fb77c2532 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:56.768727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475525f2-2526-4da6-b00a-9c0fb77c2532 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:56.768826 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475525f2-2526-4da6-b00a-9c0fb77c2532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:56.789038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-475525f2-2526-4da6-b00a-9c0fb77c2532 None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:56.790266 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-475525f2-2526-4da6-b00a-9c0fb77c2532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:56.879530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1269/2521] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:54:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:56.893670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7d0fa-fff9-44ab-ac9e-1716c3a5d949 service placement] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:56.895174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-dfc7d0fa-fff9-44ab-ac9e-1716c3a5d949 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:56.979746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7d0fa-fff9-44ab-ac9e-1716c3a5d949 service placement] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:56.985297 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4ea95-675e-4e69-83b3-804eb6db724d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:56.985941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4ea95-675e-4e69-83b3-804eb6db724d service swift] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:56.986291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4ea95-675e-4e69-83b3-804eb6db724d service swift] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:56.986566 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4ea95-675e-4e69-83b3-804eb6db724d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:56.995020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4ea95-675e-4e69-83b3-804eb6db724d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:56.996669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4ea95-675e-4e69-83b3-804eb6db724d service swift] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:57.004578 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1253/2522] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:54:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:54:57.793510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475525f2-2526-4da6-b00a-9c0fb77c2532 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:57.795583 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-475525f2-2526-4da6-b00a-9c0fb77c2532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:57.903382 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475525f2-2526-4da6-b00a-9c0fb77c2532 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:57.909701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923554c-e4b6-45f9-90db-1dc39fa19ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:57.910680 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923554c-e4b6-45f9-90db-1dc39fa19ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:57.911009 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923554c-e4b6-45f9-90db-1dc39fa19ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:57.911280 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923554c-e4b6-45f9-90db-1dc39fa19ec9 admin admin] PATH_INFO: `/v3/users/3737f6f4251f41c8af7a5bcb01a70776` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:57.916760 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923554c-e4b6-45f9-90db-1dc39fa19ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3737f6f4251f41c8af7a5bcb01a70776)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:57.919789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923554c-e4b6-45f9-90db-1dc39fa19ec9 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:57.998936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a923554c-e4b6-45f9-90db-1dc39fa19ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3737f6f4251f41c8af7a5bcb01a70776'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:58.018735 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a923554c-e4b6-45f9-90db-1dc39fa19ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3737f6f4251f41c8af7a5bcb01a70776'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:58.019505 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a923554c-e4b6-45f9-90db-1dc39fa19ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3737f6f4251f41c8af7a5bcb01a70776'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:58.020362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a923554c-e4b6-45f9-90db-1dc39fa19ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3737f6f4251f41c8af7a5bcb01a70776'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:58.030217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-a923554c-e4b6-45f9-90db-1dc39fa19ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3737f6f4251f41c8af7a5bcb01a70776'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:58.051966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1270/2523] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:54:57 2022] DELETE /identity/v3/users/3737f6f4251f41c8af7a5bcb01a70776 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:58.054496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abb6fd-7e00-49aa-8cb8-2e4962334f74 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:58.055031 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abb6fd-7e00-49aa-8cb8-2e4962334f74 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:58.055115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abb6fd-7e00-49aa-8cb8-2e4962334f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:58.086493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-97abb6fd-7e00-49aa-8cb8-2e4962334f74 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:58.089645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-97abb6fd-7e00-49aa-8cb8-2e4962334f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:58.235839 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1254/2524] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:58.247946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923554c-e4b6-45f9-90db-1dc39fa19ec9 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:58.249751 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a923554c-e4b6-45f9-90db-1dc39fa19ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:58.331520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923554c-e4b6-45f9-90db-1dc39fa19ec9 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:58.343101 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759eb23-19e9-44d7-b518-731c8afa39c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:58.344253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759eb23-19e9-44d7-b518-731c8afa39c9 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:58.344680 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759eb23-19e9-44d7-b518-731c8afa39c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:58.344964 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759eb23-19e9-44d7-b518-731c8afa39c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:58.357543 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759eb23-19e9-44d7-b518-731c8afa39c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:58.359049 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759eb23-19e9-44d7-b518-731c8afa39c9 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:58.373065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1271/2525] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:54:58.778994 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abb6fd-7e00-49aa-8cb8-2e4962334f74 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:58.781271 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-97abb6fd-7e00-49aa-8cb8-2e4962334f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:58.799278 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abb6fd-7e00-49aa-8cb8-2e4962334f74 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:58.804651 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879de083-ceff-431e-88b0-bf91e8dc95cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:58.805609 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879de083-ceff-431e-88b0-bf91e8dc95cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:58.805966 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879de083-ceff-431e-88b0-bf91e8dc95cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:58.806315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879de083-ceff-431e-88b0-bf91e8dc95cb admin admin] PATH_INFO: `/v3/projects/49e6f023ae0f420fb463780287b8caab` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:58.807644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879de083-ceff-431e-88b0-bf91e8dc95cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=49e6f023ae0f420fb463780287b8caab)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:58.809670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879de083-ceff-431e-88b0-bf91e8dc95cb admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:58.864789 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-879de083-ceff-431e-88b0-bf91e8dc95cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49e6f023ae0f420fb463780287b8caab', 'request_id': 'req-879de083-ceff-431e-88b0-bf91e8dc95cb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:58.871483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-879de083-ceff-431e-88b0-bf91e8dc95cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49e6f023ae0f420fb463780287b8caab', 'request_id': 'req-879de083-ceff-431e-88b0-bf91e8dc95cb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:58.880421 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-879de083-ceff-431e-88b0-bf91e8dc95cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49e6f023ae0f420fb463780287b8caab', 'request_id': 'req-879de083-ceff-431e-88b0-bf91e8dc95cb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:58.891816 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-879de083-ceff-431e-88b0-bf91e8dc95cb admin admin] The token cache is being invalidate because project 49e6f023ae0f420fb463780287b8caab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:54:58.892167 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-879de083-ceff-431e-88b0-bf91e8dc95cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:58.892816 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-879de083-ceff-431e-88b0-bf91e8dc95cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:54:58.894600 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1255/2526] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:54:58 2022] DELETE /identity/v3/projects/49e6f023ae0f420fb463780287b8caab => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:58.899420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc7fd7-97c7-418e-855b-eda71195391c None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:58.899967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc7fd7-97c7-418e-855b-eda71195391c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:58.899967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc7fd7-97c7-418e-855b-eda71195391c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:58.920591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d2cc7fd7-97c7-418e-855b-eda71195391c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:58.921813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d2cc7fd7-97c7-418e-855b-eda71195391c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:58.992716 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1272/2527] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:54:59.000344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc866ba-ebfd-48e6-993b-6d611c64902c None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:59.000688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc866ba-ebfd-48e6-993b-6d611c64902c None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:59.000981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc866ba-ebfd-48e6-993b-6d611c64902c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:59.047686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5bc866ba-ebfd-48e6-993b-6d611c64902c None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:59.048969 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5bc866ba-ebfd-48e6-993b-6d611c64902c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:59.057332 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1256/2528] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:59.061661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc7fd7-97c7-418e-855b-eda71195391c None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:59.073811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc7fd7-97c7-418e-855b-eda71195391c None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:59.078643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994ebdd-83bb-447a-8f2d-0cd5e5329e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:59.079342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994ebdd-83bb-447a-8f2d-0cd5e5329e9f admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:59.079630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994ebdd-83bb-447a-8f2d-0cd5e5329e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:59.079911 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994ebdd-83bb-447a-8f2d-0cd5e5329e9f admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:59.080639 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994ebdd-83bb-447a-8f2d-0cd5e5329e9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:54:59.080950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994ebdd-83bb-447a-8f2d-0cd5e5329e9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:59.082560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994ebdd-83bb-447a-8f2d-0cd5e5329e9f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:59.095664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1273/2529] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:54:59.099505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc866ba-ebfd-48e6-993b-6d611c64902c None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:59.115838 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc866ba-ebfd-48e6-993b-6d611c64902c None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:59.123400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6399dd-f098-4910-9062-0f5f22ec3f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:59.124113 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6399dd-f098-4910-9062-0f5f22ec3f06 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:59.124404 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6399dd-f098-4910-9062-0f5f22ec3f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:59.124689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6399dd-f098-4910-9062-0f5f22ec3f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:59.125403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6399dd-f098-4910-9062-0f5f22ec3f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:59.127403 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6399dd-f098-4910-9062-0f5f22ec3f06 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:59.159513 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1257/2530] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:54:59 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:54:59.164070 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994ebdd-83bb-447a-8f2d-0cd5e5329e9f admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:59.184943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994ebdd-83bb-447a-8f2d-0cd5e5329e9f admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:59.274840 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210dbab-ed8b-4e64-bdef-1598eca6186e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:59.275525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210dbab-ed8b-4e64-bdef-1598eca6186e admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:59.275813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210dbab-ed8b-4e64-bdef-1598eca6186e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:59.276099 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210dbab-ed8b-4e64-bdef-1598eca6186e admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:59.276799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210dbab-ed8b-4e64-bdef-1598eca6186e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:59.279879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210dbab-ed8b-4e64-bdef-1598eca6186e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:59.336161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1274/2531] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:54:59 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:54:59.337717 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6399dd-f098-4910-9062-0f5f22ec3f06 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:59.357893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6399dd-f098-4910-9062-0f5f22ec3f06 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:59.362770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e6fe9-351c-451b-9026-fbf196b1752a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:59.363774 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e6fe9-351c-451b-9026-fbf196b1752a admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:59.364060 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e6fe9-351c-451b-9026-fbf196b1752a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:59.364325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e6fe9-351c-451b-9026-fbf196b1752a admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:59.365067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e6fe9-351c-451b-9026-fbf196b1752a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:59.367008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e6fe9-351c-451b-9026-fbf196b1752a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:59.383066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1258/2532] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:54:59.387137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210dbab-ed8b-4e64-bdef-1598eca6186e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:59.401264 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210dbab-ed8b-4e64-bdef-1598eca6186e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:59.415273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1b51d-46c1-4f25-878d-78785a445942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:59.416918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b51d-46c1-4f25-878d-78785a445942 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:59.417253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b51d-46c1-4f25-878d-78785a445942 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:59.417722 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b51d-46c1-4f25-878d-78785a445942 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:59.418777 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1b51d-46c1-4f25-878d-78785a445942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:59.425118 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1b51d-46c1-4f25-878d-78785a445942 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:59.445891 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1275/2533] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:54:59.447170 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e6fe9-351c-451b-9026-fbf196b1752a admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:59.470460 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e6fe9-351c-451b-9026-fbf196b1752a admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:59.490420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44742350-53e2-41eb-a192-10f796067a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:59.491368 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44742350-53e2-41eb-a192-10f796067a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:59.491665 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44742350-53e2-41eb-a192-10f796067a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:59.491927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44742350-53e2-41eb-a192-10f796067a3b admin admin] PATH_INFO: `/v3/projects/d0c5a25e0bf445f38e5934c844096c6e/users/f6afd777ff0c42bfb3a544620bf1a01e/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:59.535540 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44742350-53e2-41eb-a192-10f796067a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c5a25e0bf445f38e5934c844096c6e, user_id=f6afd777ff0c42bfb3a544620bf1a01e, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:59.549194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44742350-53e2-41eb-a192-10f796067a3b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:59.615431 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1259/2534] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:54:59 2022] PUT /identity/v3/projects/d0c5a25e0bf445f38e5934c844096c6e/users/f6afd777ff0c42bfb3a544620bf1a01e/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:54:59.618495 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650a1eb-e074-41b2-9aab-91192672f7cc None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:59.618837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650a1eb-e074-41b2-9aab-91192672f7cc None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:59.619122 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650a1eb-e074-41b2-9aab-91192672f7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:59.675720 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-4650a1eb-e074-41b2-9aab-91192672f7cc None None] MFA Rules not processed for user `f6afd777ff0c42bfb3a544620bf1a01e`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:54:59.677403 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-4650a1eb-e074-41b2-9aab-91192672f7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:59.776096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1276/2535] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:54:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:54:59.783924 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44742350-53e2-41eb-a192-10f796067a3b admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:54:59.786680 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-44742350-53e2-41eb-a192-10f796067a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:54:59.946868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44742350-53e2-41eb-a192-10f796067a3b admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:54:59.951322 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f1edc-3147-4cf3-9dcb-730fcf263f2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:54:59.952053 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f1edc-3147-4cf3-9dcb-730fcf263f2b service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:54:59.952408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f1edc-3147-4cf3-9dcb-730fcf263f2b service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:54:59.952763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f1edc-3147-4cf3-9dcb-730fcf263f2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:54:59.962579 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f1edc-3147-4cf3-9dcb-730fcf263f2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:54:59.964138 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f1edc-3147-4cf3-9dcb-730fcf263f2b service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:54:59.975387 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1260/2536] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:54:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:55:10.304889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650a1eb-e074-41b2-9aab-91192672f7cc None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:10.389435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650a1eb-e074-41b2-9aab-91192672f7cc None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:10.398613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e37f2-5202-4c00-b4e9-de544ce2fc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:10.398613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e37f2-5202-4c00-b4e9-de544ce2fc4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:10.398613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e37f2-5202-4c00-b4e9-de544ce2fc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:10.398613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e37f2-5202-4c00-b4e9-de544ce2fc4f admin admin] PATH_INFO: `/v3/users/f6afd777ff0c42bfb3a544620bf1a01e` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:10.400297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e37f2-5202-4c00-b4e9-de544ce2fc4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6afd777ff0c42bfb3a544620bf1a01e)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:10.402249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e37f2-5202-4c00-b4e9-de544ce2fc4f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:10.456090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-923e37f2-5202-4c00-b4e9-de544ce2fc4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6afd777ff0c42bfb3a544620bf1a01e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:10.470155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-923e37f2-5202-4c00-b4e9-de544ce2fc4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6afd777ff0c42bfb3a544620bf1a01e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:10.470967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-923e37f2-5202-4c00-b4e9-de544ce2fc4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6afd777ff0c42bfb3a544620bf1a01e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:10.471593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-923e37f2-5202-4c00-b4e9-de544ce2fc4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6afd777ff0c42bfb3a544620bf1a01e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:10.481189 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-923e37f2-5202-4c00-b4e9-de544ce2fc4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6afd777ff0c42bfb3a544620bf1a01e'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:10.501064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1277/2537] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:55:10 2022] DELETE /identity/v3/users/f6afd777ff0c42bfb3a544620bf1a01e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:55:10.505400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cda2ba-70cc-4328-9509-8f800b9f7a03 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:10.505920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cda2ba-70cc-4328-9509-8f800b9f7a03 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:10.506024 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cda2ba-70cc-4328-9509-8f800b9f7a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:10.533842 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-56cda2ba-70cc-4328-9509-8f800b9f7a03 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:10.535991 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-56cda2ba-70cc-4328-9509-8f800b9f7a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:10.619820 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1261/2538] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:55:10.630869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e37f2-5202-4c00-b4e9-de544ce2fc4f admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:10.633111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-923e37f2-5202-4c00-b4e9-de544ce2fc4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:10.699186 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e37f2-5202-4c00-b4e9-de544ce2fc4f admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:10.705980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83f7a2-4a5b-4cb5-b4b6-a6357cb5e754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:10.706838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f7a2-4a5b-4cb5-b4b6-a6357cb5e754 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:10.707124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f7a2-4a5b-4cb5-b4b6-a6357cb5e754 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:10.707401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f7a2-4a5b-4cb5-b4b6-a6357cb5e754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:10.717176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83f7a2-4a5b-4cb5-b4b6-a6357cb5e754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:10.718810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83f7a2-4a5b-4cb5-b4b6-a6357cb5e754 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:10.732339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1278/2539] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:55:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:55:10.992351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cda2ba-70cc-4328-9509-8f800b9f7a03 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:10.993806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-56cda2ba-70cc-4328-9509-8f800b9f7a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:11.010366 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cda2ba-70cc-4328-9509-8f800b9f7a03 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:11.015241 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:11.016079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:11.016346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:11.016628 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] PATH_INFO: `/v3/projects/d0c5a25e0bf445f38e5934c844096c6e` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:11.017939 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c5a25e0bf445f38e5934c844096c6e)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:11.019911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:11.069086 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c5a25e0bf445f38e5934c844096c6e', 'request_id': 'req-84e30703-fdc2-45a1-8626-a18e415f3430'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:11.074961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c5a25e0bf445f38e5934c844096c6e', 'request_id': 'req-84e30703-fdc2-45a1-8626-a18e415f3430'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:11.082132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c5a25e0bf445f38e5934c844096c6e', 'request_id': 'req-84e30703-fdc2-45a1-8626-a18e415f3430'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:11.092972 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] The token cache is being invalidate because project d0c5a25e0bf445f38e5934c844096c6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:55:11.093260 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:11.093881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:11.095439 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1262/2540] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:55:10 2022] DELETE /identity/v3/projects/d0c5a25e0bf445f38e5934c844096c6e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:55:16.418475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f022645-803f-4334-a6b4-19ea6eb8230a None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:16.419522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f022645-803f-4334-a6b4-19ea6eb8230a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:16.419624 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f022645-803f-4334-a6b4-19ea6eb8230a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:16.439550 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-2f022645-803f-4334-a6b4-19ea6eb8230a None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:16.440775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-2f022645-803f-4334-a6b4-19ea6eb8230a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:16.510762 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1279/2541] 10.222.0.35 () {58 vars in 967 bytes} [Fri May 13 21:55:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:55:16.525653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:16.527385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:16.599953 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e30703-fdc2-45a1-8626-a18e415f3430 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:16.604886 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3c132-f7fb-4bd3-bbb7-f47875c2608c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:16.605527 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3c132-f7fb-4bd3-bbb7-f47875c2608c service swift] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:16.605812 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3c132-f7fb-4bd3-bbb7-f47875c2608c service swift] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:16.606092 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3c132-f7fb-4bd3-bbb7-f47875c2608c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:16.615844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3c132-f7fb-4bd3-bbb7-f47875c2608c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:16.617449 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3c132-f7fb-4bd3-bbb7-f47875c2608c service swift] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:16.625691 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1263/2542] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:55:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 13 21:55:17.675054 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee7952-73fe-462f-8e91-c61dfe87d075 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:17.675854 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee7952-73fe-462f-8e91-c61dfe87d075 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:17.676148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee7952-73fe-462f-8e91-c61dfe87d075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:17.695963 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-ccee7952-73fe-462f-8e91-c61dfe87d075 None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:17.697164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ccee7952-73fe-462f-8e91-c61dfe87d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:17.704394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1280/2543] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:55:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:55:17.717475 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3c132-f7fb-4bd3-bbb7-f47875c2608c service swift] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:17.727985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3c132-f7fb-4bd3-bbb7-f47875c2608c service swift] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:17.736425 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63525ba9-de72-44eb-a867-99dfc7f7371d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:17.737830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63525ba9-de72-44eb-a867-99dfc7f7371d service swift] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:17.737830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63525ba9-de72-44eb-a867-99dfc7f7371d service swift] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:17.738212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63525ba9-de72-44eb-a867-99dfc7f7371d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:17.752206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63525ba9-de72-44eb-a867-99dfc7f7371d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:17.753835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63525ba9-de72-44eb-a867-99dfc7f7371d service swift] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:17.771303 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1264/2544] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:55:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:55:18.271457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3727d54-67b6-42f8-8401-6f09378b28d0 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:18.271746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3727d54-67b6-42f8-8401-6f09378b28d0 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:18.272026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3727d54-67b6-42f8-8401-6f09378b28d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:18.291108 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-f3727d54-67b6-42f8-8401-6f09378b28d0 None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:18.292238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f3727d54-67b6-42f8-8401-6f09378b28d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:18.299323 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1281/2545] 10.222.0.35 () {58 vars in 967 bytes} [Fri May 13 21:55:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:55:18.311818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63525ba9-de72-44eb-a867-99dfc7f7371d service swift] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:18.322173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63525ba9-de72-44eb-a867-99dfc7f7371d service swift] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:18.327878 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74c310-4d51-445f-833e-58e74d954f8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:18.328529 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74c310-4d51-445f-833e-58e74d954f8f service swift] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:18.328815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74c310-4d51-445f-833e-58e74d954f8f service swift] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:18.329100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74c310-4d51-445f-833e-58e74d954f8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:18.337249 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74c310-4d51-445f-833e-58e74d954f8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:18.338862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74c310-4d51-445f-833e-58e74d954f8f service swift] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:18.346755 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1265/2546] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:55:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:55:18.528943 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fe999-f0fb-4722-bf1f-d5c2a078ff6d None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:18.529235 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fe999-f0fb-4722-bf1f-d5c2a078ff6d None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:18.529514 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fe999-f0fb-4722-bf1f-d5c2a078ff6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:18.547225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-464fe999-f0fb-4722-bf1f-d5c2a078ff6d None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:18.548337 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-464fe999-f0fb-4722-bf1f-d5c2a078ff6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:18.555394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1282/2547] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:55:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:55:18.567428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74c310-4d51-445f-833e-58e74d954f8f service swift] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:18.578166 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74c310-4d51-445f-833e-58e74d954f8f service swift] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:18.583104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527dd408-7ea1-454f-bc0e-6eea08f40514 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:18.583723 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dd408-7ea1-454f-bc0e-6eea08f40514 service swift] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:18.584011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dd408-7ea1-454f-bc0e-6eea08f40514 service swift] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:18.584288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dd408-7ea1-454f-bc0e-6eea08f40514 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:18.592058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527dd408-7ea1-454f-bc0e-6eea08f40514 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:18.593616 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527dd408-7ea1-454f-bc0e-6eea08f40514 service swift] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:18.601339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1266/2548] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:55:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:55:18.918402 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac70703-67ec-41f5-a73b-bef1d92a65fe None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:18.918916 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac70703-67ec-41f5-a73b-bef1d92a65fe None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:18.919237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac70703-67ec-41f5-a73b-bef1d92a65fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:18.936392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-9ac70703-67ec-41f5-a73b-bef1d92a65fe None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:18.937595 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9ac70703-67ec-41f5-a73b-bef1d92a65fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:18.944775 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1283/2549] 10.222.0.35 () {58 vars in 967 bytes} [Fri May 13 21:55:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:55:18.957685 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527dd408-7ea1-454f-bc0e-6eea08f40514 service swift] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:18.968702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527dd408-7ea1-454f-bc0e-6eea08f40514 service swift] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:18.973775 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3114c51-8237-4843-873e-5b69e11a7bbb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:18.974415 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3114c51-8237-4843-873e-5b69e11a7bbb service swift] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:18.974718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3114c51-8237-4843-873e-5b69e11a7bbb service swift] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:18.975014 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3114c51-8237-4843-873e-5b69e11a7bbb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:18.983218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3114c51-8237-4843-873e-5b69e11a7bbb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:18.984826 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3114c51-8237-4843-873e-5b69e11a7bbb service swift] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:18.993202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1267/2550] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:55:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:55:22.047299 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac70703-67ec-41f5-a73b-bef1d92a65fe None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:22.048796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9ac70703-67ec-41f5-a73b-bef1d92a65fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:22.115528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac70703-67ec-41f5-a73b-bef1d92a65fe None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:22.120768 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84998739-5e56-4056-8965-15d00c018ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:22.121823 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84998739-5e56-4056-8965-15d00c018ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:22.122740 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84998739-5e56-4056-8965-15d00c018ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:22.130790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84998739-5e56-4056-8965-15d00c018ae5 admin admin] PATH_INFO: `/v3/users/75a06208154149a88ca10d30520a58d5` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:22.133062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84998739-5e56-4056-8965-15d00c018ae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a06208154149a88ca10d30520a58d5)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:22.135005 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84998739-5e56-4056-8965-15d00c018ae5 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:22.194435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-84998739-5e56-4056-8965-15d00c018ae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a06208154149a88ca10d30520a58d5'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:22.207044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-84998739-5e56-4056-8965-15d00c018ae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a06208154149a88ca10d30520a58d5'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:22.207914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-84998739-5e56-4056-8965-15d00c018ae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a06208154149a88ca10d30520a58d5'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:22.208622 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-84998739-5e56-4056-8965-15d00c018ae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a06208154149a88ca10d30520a58d5'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:22.215913 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-84998739-5e56-4056-8965-15d00c018ae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a06208154149a88ca10d30520a58d5'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:22.229131 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1284/2551] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:55:22 2022] DELETE /identity/v3/users/75a06208154149a88ca10d30520a58d5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:55:22.238373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c5b19-469e-4005-86bb-8d3a16fc904a None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:22.238923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c5b19-469e-4005-86bb-8d3a16fc904a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:22.238923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c5b19-469e-4005-86bb-8d3a16fc904a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:22.259515 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-ba0c5b19-469e-4005-86bb-8d3a16fc904a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:22.260734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ba0c5b19-469e-4005-86bb-8d3a16fc904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:22.317330 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1268/2552] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:55:22.327751 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84998739-5e56-4056-8965-15d00c018ae5 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:22.329243 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-84998739-5e56-4056-8965-15d00c018ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:22.384540 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84998739-5e56-4056-8965-15d00c018ae5 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:22.388812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f607c-da08-4f9b-a542-cfbf614de697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:22.389447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f607c-da08-4f9b-a542-cfbf614de697 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:22.389731 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f607c-da08-4f9b-a542-cfbf614de697 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:22.390010 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f607c-da08-4f9b-a542-cfbf614de697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:22.398875 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f607c-da08-4f9b-a542-cfbf614de697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:22.400301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f607c-da08-4f9b-a542-cfbf614de697 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:22.409665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1285/2553] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:55:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:55:22.583033 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c5b19-469e-4005-86bb-8d3a16fc904a None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:22.584644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-ba0c5b19-469e-4005-86bb-8d3a16fc904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:22.597147 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c5b19-469e-4005-86bb-8d3a16fc904a None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:22.604193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39b449-0963-4ef9-987d-544be6c6287e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:22.605448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39b449-0963-4ef9-987d-544be6c6287e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:22.605738 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39b449-0963-4ef9-987d-544be6c6287e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:22.606064 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39b449-0963-4ef9-987d-544be6c6287e admin admin] PATH_INFO: `/v3/projects/a480dd328dbc461690b7a80095a21967` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:22.608656 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39b449-0963-4ef9-987d-544be6c6287e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a480dd328dbc461690b7a80095a21967)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:22.611042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39b449-0963-4ef9-987d-544be6c6287e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:22.661298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5c39b449-0963-4ef9-987d-544be6c6287e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a480dd328dbc461690b7a80095a21967', 'request_id': 'req-5c39b449-0963-4ef9-987d-544be6c6287e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:22.667601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5c39b449-0963-4ef9-987d-544be6c6287e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a480dd328dbc461690b7a80095a21967', 'request_id': 'req-5c39b449-0963-4ef9-987d-544be6c6287e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:22.675445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5c39b449-0963-4ef9-987d-544be6c6287e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a480dd328dbc461690b7a80095a21967', 'request_id': 'req-5c39b449-0963-4ef9-987d-544be6c6287e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:22.687566 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5c39b449-0963-4ef9-987d-544be6c6287e admin admin] The token cache is being invalidate because project a480dd328dbc461690b7a80095a21967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:55:22.688050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5c39b449-0963-4ef9-987d-544be6c6287e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:22.688740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5c39b449-0963-4ef9-987d-544be6c6287e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:22.690365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1269/2554] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:55:22 2022] DELETE /identity/v3/projects/a480dd328dbc461690b7a80095a21967 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:55:26.372390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f607c-da08-4f9b-a542-cfbf614de697 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:26.373807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b89f607c-da08-4f9b-a542-cfbf614de697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:26.386246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f607c-da08-4f9b-a542-cfbf614de697 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:26.439398 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e880836-ac61-4c02-a266-c8cffa132444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:26.440252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e880836-ac61-4c02-a266-c8cffa132444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:26.440535 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e880836-ac61-4c02-a266-c8cffa132444 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:26.440813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e880836-ac61-4c02-a266-c8cffa132444 admin admin] PATH_INFO: `/v3/users/838b4334782b418f9ac593d04fda373f` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:26.441979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e880836-ac61-4c02-a266-c8cffa132444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=838b4334782b418f9ac593d04fda373f)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:26.443925 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e880836-ac61-4c02-a266-c8cffa132444 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:26.488667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9e880836-ac61-4c02-a266-c8cffa132444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '838b4334782b418f9ac593d04fda373f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:26.500082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9e880836-ac61-4c02-a266-c8cffa132444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '838b4334782b418f9ac593d04fda373f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:26.500745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9e880836-ac61-4c02-a266-c8cffa132444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '838b4334782b418f9ac593d04fda373f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:26.501349 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9e880836-ac61-4c02-a266-c8cffa132444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '838b4334782b418f9ac593d04fda373f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:26.507994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9e880836-ac61-4c02-a266-c8cffa132444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '838b4334782b418f9ac593d04fda373f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:26.520790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1286/2555] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:55:26 2022] DELETE /identity/v3/users/838b4334782b418f9ac593d04fda373f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:55:26.525218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ecafb-a5d0-443b-b4b7-91b8a6c24244 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:26.525681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ecafb-a5d0-443b-b4b7-91b8a6c24244 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:26.525840 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ecafb-a5d0-443b-b4b7-91b8a6c24244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:26.544746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-064ecafb-a5d0-443b-b4b7-91b8a6c24244 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:26.545998 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-064ecafb-a5d0-443b-b4b7-91b8a6c24244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:26.602310 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1270/2556] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:55:26.611839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e880836-ac61-4c02-a266-c8cffa132444 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:26.613499 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9e880836-ac61-4c02-a266-c8cffa132444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:26.669532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e880836-ac61-4c02-a266-c8cffa132444 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:26.673172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f9a4b-71e2-4245-a034-017a83f10b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:26.673804 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f9a4b-71e2-4245-a034-017a83f10b6b service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:26.674088 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f9a4b-71e2-4245-a034-017a83f10b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:26.674366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f9a4b-71e2-4245-a034-017a83f10b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:26.683171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f9a4b-71e2-4245-a034-017a83f10b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:26.685106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f9a4b-71e2-4245-a034-017a83f10b6b service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:26.694559 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1287/2557] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:55:26.955174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ecafb-a5d0-443b-b4b7-91b8a6c24244 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:26.956656 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-064ecafb-a5d0-443b-b4b7-91b8a6c24244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:26.969233 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ecafb-a5d0-443b-b4b7-91b8a6c24244 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:26.978968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528872b-c178-4ab4-92c6-5fa55889dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:26.979866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528872b-c178-4ab4-92c6-5fa55889dcf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:26.980151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528872b-c178-4ab4-92c6-5fa55889dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:26.980424 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528872b-c178-4ab4-92c6-5fa55889dcf2 admin admin] PATH_INFO: `/v3/projects/afd714bddd13492cb9bfb9eb20c40e98` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:26.982144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528872b-c178-4ab4-92c6-5fa55889dcf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afd714bddd13492cb9bfb9eb20c40e98)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:26.984089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528872b-c178-4ab4-92c6-5fa55889dcf2 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:27.028344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e528872b-c178-4ab4-92c6-5fa55889dcf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afd714bddd13492cb9bfb9eb20c40e98', 'request_id': 'req-e528872b-c178-4ab4-92c6-5fa55889dcf2'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:27.034226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e528872b-c178-4ab4-92c6-5fa55889dcf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afd714bddd13492cb9bfb9eb20c40e98', 'request_id': 'req-e528872b-c178-4ab4-92c6-5fa55889dcf2'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:27.042001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e528872b-c178-4ab4-92c6-5fa55889dcf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afd714bddd13492cb9bfb9eb20c40e98', 'request_id': 'req-e528872b-c178-4ab4-92c6-5fa55889dcf2'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:27.052198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e528872b-c178-4ab4-92c6-5fa55889dcf2 admin admin] The token cache is being invalidate because project afd714bddd13492cb9bfb9eb20c40e98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:55:27.052484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e528872b-c178-4ab4-92c6-5fa55889dcf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:27.053091 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-e528872b-c178-4ab4-92c6-5fa55889dcf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:27.054619 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1271/2558] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:55:26 2022] DELETE /identity/v3/projects/afd714bddd13492cb9bfb9eb20c40e98 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:55:27.060001 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfaf64-70ae-4092-9f17-d9dae0ced7f8 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:27.060322 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfaf64-70ae-4092-9f17-d9dae0ced7f8 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:27.060600 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfaf64-70ae-4092-9f17-d9dae0ced7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:27.077501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-99bfaf64-70ae-4092-9f17-d9dae0ced7f8 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:27.078683 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-99bfaf64-70ae-4092-9f17-d9dae0ced7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:27.135905 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1288/2559] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:55:27.144525 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ece07-29c9-4756-be57-d933e26aac8d None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:27.144810 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ece07-29c9-4756-be57-d933e26aac8d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:27.145086 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ece07-29c9-4756-be57-d933e26aac8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:27.163195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-bc2ece07-29c9-4756-be57-d933e26aac8d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:27.164389 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-bc2ece07-29c9-4756-be57-d933e26aac8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:27.173569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1272/2560] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:55:27.177921 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfaf64-70ae-4092-9f17-d9dae0ced7f8 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:27.188261 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfaf64-70ae-4092-9f17-d9dae0ced7f8 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:27.192888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e65af6-7721-4064-afa6-fd3086f4aeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:27.193542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e65af6-7721-4064-afa6-fd3086f4aeaf admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:27.193827 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e65af6-7721-4064-afa6-fd3086f4aeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:27.194105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e65af6-7721-4064-afa6-fd3086f4aeaf admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:27.194868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e65af6-7721-4064-afa6-fd3086f4aeaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:55:27.195176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e65af6-7721-4064-afa6-fd3086f4aeaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:27.196791 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e65af6-7721-4064-afa6-fd3086f4aeaf admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:27.208474 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1289/2561] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:55:27.212623 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ece07-29c9-4756-be57-d933e26aac8d None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:27.221599 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ece07-29c9-4756-be57-d933e26aac8d None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:27.226145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14673b-932e-4556-9070-93d7ecf17f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:27.226814 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14673b-932e-4556-9070-93d7ecf17f5c admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:27.227097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14673b-932e-4556-9070-93d7ecf17f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:27.227371 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14673b-932e-4556-9070-93d7ecf17f5c admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:27.228049 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14673b-932e-4556-9070-93d7ecf17f5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:27.229927 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14673b-932e-4556-9070-93d7ecf17f5c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:27.256387 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1273/2562] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:55:27 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:55:27.260354 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e65af6-7721-4064-afa6-fd3086f4aeaf admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:27.270094 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e65af6-7721-4064-afa6-fd3086f4aeaf admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:27.323745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230ef07-828e-4aaf-9d9c-4444c1cdb31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:27.324403 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230ef07-828e-4aaf-9d9c-4444c1cdb31f admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:27.324687 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230ef07-828e-4aaf-9d9c-4444c1cdb31f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:27.324979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230ef07-828e-4aaf-9d9c-4444c1cdb31f admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:27.325671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230ef07-828e-4aaf-9d9c-4444c1cdb31f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:27.327633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230ef07-828e-4aaf-9d9c-4444c1cdb31f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:27.347320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1290/2563] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:55:27 2022] POST /identity/v3/users => generated 287 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:55:27.351277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14673b-932e-4556-9070-93d7ecf17f5c admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:27.361626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14673b-932e-4556-9070-93d7ecf17f5c admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:27.366418 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7808b6-800b-40d9-a560-75a7fb5dd929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:27.367097 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7808b6-800b-40d9-a560-75a7fb5dd929 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:27.367378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7808b6-800b-40d9-a560-75a7fb5dd929 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:27.367652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7808b6-800b-40d9-a560-75a7fb5dd929 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:27.368369 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7808b6-800b-40d9-a560-75a7fb5dd929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:27.369940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7808b6-800b-40d9-a560-75a7fb5dd929 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:27.379850 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1274/2564] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:55:27.383386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230ef07-828e-4aaf-9d9c-4444c1cdb31f admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:27.393105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230ef07-828e-4aaf-9d9c-4444c1cdb31f admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:27.397683 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03af718-06df-426b-ba81-008f2b7ca349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:27.398318 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03af718-06df-426b-ba81-008f2b7ca349 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:27.398604 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03af718-06df-426b-ba81-008f2b7ca349 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:27.398988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03af718-06df-426b-ba81-008f2b7ca349 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:27.399702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03af718-06df-426b-ba81-008f2b7ca349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:27.401319 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03af718-06df-426b-ba81-008f2b7ca349 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:27.412160 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1291/2565] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:55:27.416206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7808b6-800b-40d9-a560-75a7fb5dd929 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:27.426316 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7808b6-800b-40d9-a560-75a7fb5dd929 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:27.430884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72039d46-26ac-442c-82f4-6d0417e8bddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:27.431739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72039d46-26ac-442c-82f4-6d0417e8bddc admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:27.432019 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72039d46-26ac-442c-82f4-6d0417e8bddc admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:27.432294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72039d46-26ac-442c-82f4-6d0417e8bddc admin admin] PATH_INFO: `/v3/projects/2c9071329a44416d82adc568d4db5cca/users/fd54c69c87274f4387ea0f9d676c524c/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:27.445987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72039d46-26ac-442c-82f4-6d0417e8bddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c9071329a44416d82adc568d4db5cca, user_id=fd54c69c87274f4387ea0f9d676c524c, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:27.448632 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72039d46-26ac-442c-82f4-6d0417e8bddc admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:27.461256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1275/2566] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:55:27 2022] PUT /identity/v3/projects/2c9071329a44416d82adc568d4db5cca/users/fd54c69c87274f4387ea0f9d676c524c/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:55:27.471711 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254862c0-bb1e-499b-9f8a-bafa5c4e5982 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:27.472025 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254862c0-bb1e-499b-9f8a-bafa5c4e5982 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:27.472335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254862c0-bb1e-499b-9f8a-bafa5c4e5982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:27.499776 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-254862c0-bb1e-499b-9f8a-bafa5c4e5982 None None] MFA Rules not processed for user `fd54c69c87274f4387ea0f9d676c524c`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:27.500982 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-254862c0-bb1e-499b-9f8a-bafa5c4e5982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:27.561367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1292/2567] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:55:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:55:27.572437 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72039d46-26ac-442c-82f4-6d0417e8bddc admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:27.574191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-72039d46-26ac-442c-82f4-6d0417e8bddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:27.628747 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72039d46-26ac-442c-82f4-6d0417e8bddc admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:27.632464 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4289b5-cfda-4880-b207-4eb631f96757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:27.633110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4289b5-cfda-4880-b207-4eb631f96757 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:27.633391 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4289b5-cfda-4880-b207-4eb631f96757 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:27.633698 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4289b5-cfda-4880-b207-4eb631f96757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:27.641878 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4289b5-cfda-4880-b207-4eb631f96757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:27.643345 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4289b5-cfda-4880-b207-4eb631f96757 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:27.652316 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1276/2568] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:55:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:55:27.677291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254862c0-bb1e-499b-9f8a-bafa5c4e5982 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:27.737084 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254862c0-bb1e-499b-9f8a-bafa5c4e5982 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:27.741881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51e6e6-f90d-46b0-937a-730034c03000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:27.742762 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e6e6-f90d-46b0-937a-730034c03000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:27.743052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e6e6-f90d-46b0-937a-730034c03000 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:27.743330 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e6e6-f90d-46b0-937a-730034c03000 admin admin] PATH_INFO: `/v3/users/fd54c69c87274f4387ea0f9d676c524c` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:27.744524 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51e6e6-f90d-46b0-937a-730034c03000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd54c69c87274f4387ea0f9d676c524c)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:27.746453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51e6e6-f90d-46b0-937a-730034c03000 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:27.794231 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-de51e6e6-f90d-46b0-937a-730034c03000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd54c69c87274f4387ea0f9d676c524c'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:27.805718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-de51e6e6-f90d-46b0-937a-730034c03000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd54c69c87274f4387ea0f9d676c524c'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:27.806416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-de51e6e6-f90d-46b0-937a-730034c03000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd54c69c87274f4387ea0f9d676c524c'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:27.807058 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-de51e6e6-f90d-46b0-937a-730034c03000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd54c69c87274f4387ea0f9d676c524c'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:27.813886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-de51e6e6-f90d-46b0-937a-730034c03000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd54c69c87274f4387ea0f9d676c524c'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:27.826971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1293/2569] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:55:27 2022] DELETE /identity/v3/users/fd54c69c87274f4387ea0f9d676c524c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:55:27.831440 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bb928-c915-474c-9f43-3eb61dfcbdb9 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:27.831756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bb928-c915-474c-9f43-3eb61dfcbdb9 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:27.832037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bb928-c915-474c-9f43-3eb61dfcbdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:27.849220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-196bb928-c915-474c-9f43-3eb61dfcbdb9 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:27.850406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-196bb928-c915-474c-9f43-3eb61dfcbdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:27.907339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1277/2570] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:55:27.917353 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51e6e6-f90d-46b0-937a-730034c03000 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:27.919024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-de51e6e6-f90d-46b0-937a-730034c03000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:27.975780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51e6e6-f90d-46b0-937a-730034c03000 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:27.979441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd587e-7875-4821-8602-f430c2e5e534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:27.980077 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd587e-7875-4821-8602-f430c2e5e534 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:27.980359 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd587e-7875-4821-8602-f430c2e5e534 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:27.980633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd587e-7875-4821-8602-f430c2e5e534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:27.989636 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd587e-7875-4821-8602-f430c2e5e534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:27.991057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd587e-7875-4821-8602-f430c2e5e534 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:28.000228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1294/2571] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:55:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:55:28.256866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bb928-c915-474c-9f43-3eb61dfcbdb9 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:28.258545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-196bb928-c915-474c-9f43-3eb61dfcbdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:28.277315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bb928-c915-474c-9f43-3eb61dfcbdb9 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:28.282463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337129c-491f-49c1-a70a-a9a9bdb6bc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:28.283273 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337129c-491f-49c1-a70a-a9a9bdb6bc31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:28.283561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337129c-491f-49c1-a70a-a9a9bdb6bc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:28.283841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337129c-491f-49c1-a70a-a9a9bdb6bc31 admin admin] PATH_INFO: `/v3/projects/2c9071329a44416d82adc568d4db5cca` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:28.285035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337129c-491f-49c1-a70a-a9a9bdb6bc31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c9071329a44416d82adc568d4db5cca)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:28.287014 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337129c-491f-49c1-a70a-a9a9bdb6bc31 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:28.332558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0337129c-491f-49c1-a70a-a9a9bdb6bc31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c9071329a44416d82adc568d4db5cca', 'request_id': 'req-0337129c-491f-49c1-a70a-a9a9bdb6bc31'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:28.338419 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0337129c-491f-49c1-a70a-a9a9bdb6bc31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c9071329a44416d82adc568d4db5cca', 'request_id': 'req-0337129c-491f-49c1-a70a-a9a9bdb6bc31'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:28.344773 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0337129c-491f-49c1-a70a-a9a9bdb6bc31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c9071329a44416d82adc568d4db5cca', 'request_id': 'req-0337129c-491f-49c1-a70a-a9a9bdb6bc31'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:28.354893 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0337129c-491f-49c1-a70a-a9a9bdb6bc31 admin admin] The token cache is being invalidate because project 2c9071329a44416d82adc568d4db5cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:55:28.355184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0337129c-491f-49c1-a70a-a9a9bdb6bc31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:28.355820 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0337129c-491f-49c1-a70a-a9a9bdb6bc31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:28.357390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1278/2572] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:55:28 2022] DELETE /identity/v3/projects/2c9071329a44416d82adc568d4db5cca => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:55:28.362316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2878a-09f5-4cc3-82f5-55e926140ac6 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:28.362910 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2878a-09f5-4cc3-82f5-55e926140ac6 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:28.363197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2878a-09f5-4cc3-82f5-55e926140ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:28.380578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-b1d2878a-09f5-4cc3-82f5-55e926140ac6 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:28.381778 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b1d2878a-09f5-4cc3-82f5-55e926140ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:28.439135 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1295/2573] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:55:28.446891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1a1b4-b574-42ab-893d-da3f920685d2 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:28.447184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1a1b4-b574-42ab-893d-da3f920685d2 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:28.447466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1a1b4-b574-42ab-893d-da3f920685d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:28.465886 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-43b1a1b4-b574-42ab-893d-da3f920685d2 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:28.467138 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-43b1a1b4-b574-42ab-893d-da3f920685d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:28.474955 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1279/2574] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:55:28.479136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2878a-09f5-4cc3-82f5-55e926140ac6 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:28.489289 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2878a-09f5-4cc3-82f5-55e926140ac6 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:28.493928 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b673f8-9804-4f08-8f25-06ccbea10219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:28.494574 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b673f8-9804-4f08-8f25-06ccbea10219 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:28.494913 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b673f8-9804-4f08-8f25-06ccbea10219 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:28.495187 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b673f8-9804-4f08-8f25-06ccbea10219 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:28.495904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b673f8-9804-4f08-8f25-06ccbea10219 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:55:28.496210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b673f8-9804-4f08-8f25-06ccbea10219 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:28.498431 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b673f8-9804-4f08-8f25-06ccbea10219 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:28.509751 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1296/2575] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:55:28.513864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1a1b4-b574-42ab-893d-da3f920685d2 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:28.523112 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1a1b4-b574-42ab-893d-da3f920685d2 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:28.527760 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a0032-752d-49a7-93fc-805571faa92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:28.528391 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a0032-752d-49a7-93fc-805571faa92d admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:28.528679 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a0032-752d-49a7-93fc-805571faa92d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:28.528960 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a0032-752d-49a7-93fc-805571faa92d admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:28.529645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a0032-752d-49a7-93fc-805571faa92d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:28.531586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a0032-752d-49a7-93fc-805571faa92d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:28.556517 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1280/2576] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:55:28 2022] POST /identity/v3/projects => generated 391 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:55:28.561277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b673f8-9804-4f08-8f25-06ccbea10219 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:28.571003 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b673f8-9804-4f08-8f25-06ccbea10219 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:28.625250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c28e92-2ec8-4839-b138-6f1055693a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:28.625899 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c28e92-2ec8-4839-b138-6f1055693a60 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:28.626183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c28e92-2ec8-4839-b138-6f1055693a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:28.626460 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c28e92-2ec8-4839-b138-6f1055693a60 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:28.627185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c28e92-2ec8-4839-b138-6f1055693a60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:28.629104 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c28e92-2ec8-4839-b138-6f1055693a60 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:28.648196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1297/2577] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:55:28 2022] POST /identity/v3/users => generated 296 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:55:28.652302 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a0032-752d-49a7-93fc-805571faa92d admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:28.662480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a0032-752d-49a7-93fc-805571faa92d admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:28.667051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23d73e-8447-43e0-bd07-e2f1feb68a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:28.667707 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23d73e-8447-43e0-bd07-e2f1feb68a77 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:28.667994 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23d73e-8447-43e0-bd07-e2f1feb68a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:28.668273 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23d73e-8447-43e0-bd07-e2f1feb68a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:28.669530 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23d73e-8447-43e0-bd07-e2f1feb68a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:28.671169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23d73e-8447-43e0-bd07-e2f1feb68a77 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:28.681058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1281/2578] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:55:28.685292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c28e92-2ec8-4839-b138-6f1055693a60 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:28.694922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c28e92-2ec8-4839-b138-6f1055693a60 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:28.699444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7b13c-d598-41c1-b776-c997ca54aeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:28.700081 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7b13c-d598-41c1-b776-c997ca54aeab admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:28.700361 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7b13c-d598-41c1-b776-c997ca54aeab admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:28.700641 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7b13c-d598-41c1-b776-c997ca54aeab admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:28.701352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7b13c-d598-41c1-b776-c997ca54aeab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:28.702962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7b13c-d598-41c1-b776-c997ca54aeab admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:28.712517 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1298/2579] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:55:28.716280 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23d73e-8447-43e0-bd07-e2f1feb68a77 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:28.725847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23d73e-8447-43e0-bd07-e2f1feb68a77 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:28.730376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace318-00ec-43e1-83d2-029ed0d694d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:28.731769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace318-00ec-43e1-83d2-029ed0d694d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:28.732050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace318-00ec-43e1-83d2-029ed0d694d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:28.732325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace318-00ec-43e1-83d2-029ed0d694d6 admin admin] PATH_INFO: `/v3/projects/77d3c3ed33204ee697f850287e1d5ab5/users/bcb3bb7747e44fc69dd289bebd1ba7e4/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:28.747964 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ace318-00ec-43e1-83d2-029ed0d694d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d3c3ed33204ee697f850287e1d5ab5, user_id=bcb3bb7747e44fc69dd289bebd1ba7e4, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:28.750589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ace318-00ec-43e1-83d2-029ed0d694d6 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:28.763227 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1282/2580] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:55:28 2022] PUT /identity/v3/projects/77d3c3ed33204ee697f850287e1d5ab5/users/bcb3bb7747e44fc69dd289bebd1ba7e4/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:55:28.774265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcdeef-d280-4d2b-9ad6-c9646b5c8365 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:28.774568 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcdeef-d280-4d2b-9ad6-c9646b5c8365 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:28.774883 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcdeef-d280-4d2b-9ad6-c9646b5c8365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:28.801234 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-32dcdeef-d280-4d2b-9ad6-c9646b5c8365 None None] MFA Rules not processed for user `bcb3bb7747e44fc69dd289bebd1ba7e4`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:28.802440 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-32dcdeef-d280-4d2b-9ad6-c9646b5c8365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:28.865816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1299/2581] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:55:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:55:28.876447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace318-00ec-43e1-83d2-029ed0d694d6 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:28.878168 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-31ace318-00ec-43e1-83d2-029ed0d694d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:28.933020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace318-00ec-43e1-83d2-029ed0d694d6 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:28.937433 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370b79c-22d9-4efc-a542-4337d59ff252 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:28.938134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370b79c-22d9-4efc-a542-4337d59ff252 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:28.938423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370b79c-22d9-4efc-a542-4337d59ff252 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:28.938724 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370b79c-22d9-4efc-a542-4337d59ff252 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:28.947118 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370b79c-22d9-4efc-a542-4337d59ff252 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:28.948773 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370b79c-22d9-4efc-a542-4337d59ff252 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:28.958091 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1283/2582] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:55:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:55:31.408765 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dcdeef-d280-4d2b-9ad6-c9646b5c8365 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:31.410221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-32dcdeef-d280-4d2b-9ad6-c9646b5c8365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:31.473834 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dcdeef-d280-4d2b-9ad6-c9646b5c8365 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:31.477698 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a013a10-c8bb-4dd2-9f96-6310629e6b23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:31.478106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013a10-c8bb-4dd2-9f96-6310629e6b23 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:31.478386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013a10-c8bb-4dd2-9f96-6310629e6b23 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:31.478661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013a10-c8bb-4dd2-9f96-6310629e6b23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:31.480566 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-8a013a10-c8bb-4dd2-9f96-6310629e6b23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:31.558268 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a013a10-c8bb-4dd2-9f96-6310629e6b23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:31.560454 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a013a10-c8bb-4dd2-9f96-6310629e6b23 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:31.570122 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1300/2583] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:55:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:55:47.175821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370b79c-22d9-4efc-a542-4337d59ff252 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:47.236363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370b79c-22d9-4efc-a542-4337d59ff252 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:47.241149 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a02df2-37b4-4ee6-93e2-082e439e6661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:47.242011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a02df2-37b4-4ee6-93e2-082e439e6661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:47.242300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a02df2-37b4-4ee6-93e2-082e439e6661 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:47.242581 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a02df2-37b4-4ee6-93e2-082e439e6661 admin admin] PATH_INFO: `/v3/users/bcb3bb7747e44fc69dd289bebd1ba7e4` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:47.243811 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a02df2-37b4-4ee6-93e2-082e439e6661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb3bb7747e44fc69dd289bebd1ba7e4)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:47.245729 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a02df2-37b4-4ee6-93e2-082e439e6661 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:47.292622 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b0a02df2-37b4-4ee6-93e2-082e439e6661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb3bb7747e44fc69dd289bebd1ba7e4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:47.303797 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b0a02df2-37b4-4ee6-93e2-082e439e6661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb3bb7747e44fc69dd289bebd1ba7e4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:47.304469 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b0a02df2-37b4-4ee6-93e2-082e439e6661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb3bb7747e44fc69dd289bebd1ba7e4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:47.305063 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b0a02df2-37b4-4ee6-93e2-082e439e6661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb3bb7747e44fc69dd289bebd1ba7e4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:47.312221 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-b0a02df2-37b4-4ee6-93e2-082e439e6661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb3bb7747e44fc69dd289bebd1ba7e4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:47.325426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1284/2584] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:55:47 2022] DELETE /identity/v3/users/bcb3bb7747e44fc69dd289bebd1ba7e4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:55:47.329815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398dcc76-be20-4c5e-962b-5654888c0268 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:47.330293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398dcc76-be20-4c5e-962b-5654888c0268 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:47.330494 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398dcc76-be20-4c5e-962b-5654888c0268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:47.346967 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-398dcc76-be20-4c5e-962b-5654888c0268 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:55:47.348175 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-398dcc76-be20-4c5e-962b-5654888c0268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:47.406746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1301/2585] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:55:47.416015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a02df2-37b4-4ee6-93e2-082e439e6661 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:47.417447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b0a02df2-37b4-4ee6-93e2-082e439e6661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:47.473598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a02df2-37b4-4ee6-93e2-082e439e6661 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:47.477215 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9e4e1-0ca5-4a66-8f45-4cfd7fec98e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:47.477852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9e4e1-0ca5-4a66-8f45-4cfd7fec98e1 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:47.478134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9e4e1-0ca5-4a66-8f45-4cfd7fec98e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:47.478409 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9e4e1-0ca5-4a66-8f45-4cfd7fec98e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:47.487720 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9e4e1-0ca5-4a66-8f45-4cfd7fec98e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:47.489140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9e4e1-0ca5-4a66-8f45-4cfd7fec98e1 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:47.499007 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1285/2586] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:55:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:55:47.748951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398dcc76-be20-4c5e-962b-5654888c0268 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:55:47.750396 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-398dcc76-be20-4c5e-962b-5654888c0268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:55:47.762421 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398dcc76-be20-4c5e-962b-5654888c0268 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:55:47.767057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:55:47.767920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:55:47.768206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:55:47.768480 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b admin admin] PATH_INFO: `/v3/projects/77d3c3ed33204ee697f850287e1d5ab5` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:55:47.769664 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d3c3ed33204ee697f850287e1d5ab5)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:55:47.771629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:55:47.818247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d3c3ed33204ee697f850287e1d5ab5', 'request_id': 'req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:47.824251 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d3c3ed33204ee697f850287e1d5ab5', 'request_id': 'req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:47.835045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d3c3ed33204ee697f850287e1d5ab5', 'request_id': 'req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:47.845027 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b admin admin] The token cache is being invalidate because project 77d3c3ed33204ee697f850287e1d5ab5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:55:47.845315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:47.845927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b3847c31-60cd-4d0c-b3b6-f8297d021e6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:55:47.847494 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1302/2587] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:55:47 2022] DELETE /identity/v3/projects/77d3c3ed33204ee697f850287e1d5ab5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:56:08.355707 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a130b-124f-468d-b41e-6eba2501ec3d None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:56:08.356382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a130b-124f-468d-b41e-6eba2501ec3d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:56:08.356382 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a130b-124f-468d-b41e-6eba2501ec3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:56:08.374208 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-fc9a130b-124f-468d-b41e-6eba2501ec3d None None] MFA Rules not processed for user `0a1425d092844a2e9b8a917dc9981b32`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:56:08.375719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-fc9a130b-124f-468d-b41e-6eba2501ec3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:56:08.436007 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1286/2588] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:56:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:56:18.028857 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2053d-7d46-41b7-a81a-155d7be59545 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:56:18.029374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2053d-7d46-41b7-a81a-155d7be59545 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:56:18.029474 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2053d-7d46-41b7-a81a-155d7be59545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:56:18.047263 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7db2053d-7d46-41b7-a81a-155d7be59545 None None] MFA Rules not processed for user `6b11a3de8b71482594a04ab7e53a94bd`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:56:18.048457 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7db2053d-7d46-41b7-a81a-155d7be59545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:56:18.104128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1303/2589] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:56:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:56:26.739026 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a411a-9378-4360-b93d-7070ffa88108 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:56:26.739480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a411a-9378-4360-b93d-7070ffa88108 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:56:26.739572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a411a-9378-4360-b93d-7070ffa88108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:56:26.757279 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-8c3a411a-9378-4360-b93d-7070ffa88108 None None] MFA Rules not processed for user `c3a0de9ca12746be893b97d3cd11b826`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:56:26.758570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-8c3a411a-9378-4360-b93d-7070ffa88108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:56:26.810539 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1287/2590] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:56:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:56:38.067530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2053d-7d46-41b7-a81a-155d7be59545 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:56:38.068915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7db2053d-7d46-41b7-a81a-155d7be59545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:56:38.130167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2053d-7d46-41b7-a81a-155d7be59545 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:56:38.133738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6c28c-d10a-40cc-9db3-baaf3da417c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:56:38.134375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6c28c-d10a-40cc-9db3-baaf3da417c2 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:56:38.134653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6c28c-d10a-40cc-9db3-baaf3da417c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:56:38.134954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6c28c-d10a-40cc-9db3-baaf3da417c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:56:38.136676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c7c6c28c-d10a-40cc-9db3-baaf3da417c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:56:38.217824 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6c28c-d10a-40cc-9db3-baaf3da417c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:56:38.219320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6c28c-d10a-40cc-9db3-baaf3da417c2 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:56:38.239684 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1304/2591] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:56:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:56:39.357452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a411a-9378-4360-b93d-7070ffa88108 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:56:39.367067 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a411a-9378-4360-b93d-7070ffa88108 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:56:39.370572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559a5a2-27fa-46e2-83c4-2d571a9a1489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:56:39.371237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559a5a2-27fa-46e2-83c4-2d571a9a1489 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:56:39.371520 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559a5a2-27fa-46e2-83c4-2d571a9a1489 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:56:39.372362 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559a5a2-27fa-46e2-83c4-2d571a9a1489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:56:39.374113 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4559a5a2-27fa-46e2-83c4-2d571a9a1489 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:56:39.387642 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559a5a2-27fa-46e2-83c4-2d571a9a1489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:56:39.389140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559a5a2-27fa-46e2-83c4-2d571a9a1489 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:56:39.400074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1288/2592] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:56:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:57:04.332980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6c28c-d10a-40cc-9db3-baaf3da417c2 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:04.334470 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c7c6c28c-d10a-40cc-9db3-baaf3da417c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:04.348643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6c28c-d10a-40cc-9db3-baaf3da417c2 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:04.354161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b327b95-3cd4-4954-abb7-a38ae3c2bc2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:04.354861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b327b95-3cd4-4954-abb7-a38ae3c2bc2a service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:04.355149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b327b95-3cd4-4954-abb7-a38ae3c2bc2a service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:04.355429 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b327b95-3cd4-4954-abb7-a38ae3c2bc2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:04.357231 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-0b327b95-3cd4-4954-abb7-a38ae3c2bc2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:04.368172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b327b95-3cd4-4954-abb7-a38ae3c2bc2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:04.369756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b327b95-3cd4-4954-abb7-a38ae3c2bc2a service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:04.378654 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1305/2593] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:57:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:57:04.489674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559a5a2-27fa-46e2-83c4-2d571a9a1489 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:04.491160 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4559a5a2-27fa-46e2-83c4-2d571a9a1489 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:04.548248 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559a5a2-27fa-46e2-83c4-2d571a9a1489 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:04.552117 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fcc6a-9e4b-4bbc-aede-22ffc96cff26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:04.552769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fcc6a-9e4b-4bbc-aede-22ffc96cff26 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:04.553054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fcc6a-9e4b-4bbc-aede-22ffc96cff26 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:04.553332 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fcc6a-9e4b-4bbc-aede-22ffc96cff26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:04.555157 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-bb0fcc6a-9e4b-4bbc-aede-22ffc96cff26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:04.566455 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fcc6a-9e4b-4bbc-aede-22ffc96cff26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:04.567940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fcc6a-9e4b-4bbc-aede-22ffc96cff26 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:04.577513 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1289/2594] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:57:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:57:05.918126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b327b95-3cd4-4954-abb7-a38ae3c2bc2a service nova] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:05.920273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-0b327b95-3cd4-4954-abb7-a38ae3c2bc2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:05.942599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b327b95-3cd4-4954-abb7-a38ae3c2bc2a service nova] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:05.950124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dceaa-d25a-4ca0-af92-a0a1ded33a11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:05.951071 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dceaa-d25a-4ca0-af92-a0a1ded33a11 service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:05.951578 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dceaa-d25a-4ca0-af92-a0a1ded33a11 service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:05.958750 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dceaa-d25a-4ca0-af92-a0a1ded33a11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:05.963352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-588dceaa-d25a-4ca0-af92-a0a1ded33a11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:05.980923 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dceaa-d25a-4ca0-af92-a0a1ded33a11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:05.982959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dceaa-d25a-4ca0-af92-a0a1ded33a11 service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:06.009632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1306/2595] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:57:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:57:06.112089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fcc6a-9e4b-4bbc-aede-22ffc96cff26 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:06.123555 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fcc6a-9e4b-4bbc-aede-22ffc96cff26 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:06.128311 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0b0d2-d6a9-42a7-ba00-8bfde7c64ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:06.128994 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0b0d2-d6a9-42a7-ba00-8bfde7c64ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:06.129323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0b0d2-d6a9-42a7-ba00-8bfde7c64ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:06.129645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0b0d2-d6a9-42a7-ba00-8bfde7c64ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:06.138336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f7a0b0d2-d6a9-42a7-ba00-8bfde7c64ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:06.172050 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0b0d2-d6a9-42a7-ba00-8bfde7c64ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:06.173713 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0b0d2-d6a9-42a7-ba00-8bfde7c64ba2 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:06.193649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1290/2596] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:57:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:57:16.614675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fb97e-ff55-4028-a9c2-b9a7516862d8 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:16.615186 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fb97e-ff55-4028-a9c2-b9a7516862d8 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:16.615292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fb97e-ff55-4028-a9c2-b9a7516862d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:16.634184 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-327fb97e-ff55-4028-a9c2-b9a7516862d8 None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:57:16.635475 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-327fb97e-ff55-4028-a9c2-b9a7516862d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:16.692428 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1307/2597] 10.222.0.35 () {58 vars in 967 bytes} [Fri May 13 21:57:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:57:16.737342 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0b0d2-d6a9-42a7-ba00-8bfde7c64ba2 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:16.738783 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f7a0b0d2-d6a9-42a7-ba00-8bfde7c64ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:16.805065 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0b0d2-d6a9-42a7-ba00-8bfde7c64ba2 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:16.810089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551e7cc-f51f-435d-b468-7109df8ae2ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:16.810750 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551e7cc-f51f-435d-b468-7109df8ae2ac service swift] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:16.811041 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551e7cc-f51f-435d-b468-7109df8ae2ac service swift] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:16.811317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551e7cc-f51f-435d-b468-7109df8ae2ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:16.819765 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551e7cc-f51f-435d-b468-7109df8ae2ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:16.821345 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551e7cc-f51f-435d-b468-7109df8ae2ac service swift] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:16.829115 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1291/2598] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:57:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:57:24.426224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fb97e-ff55-4028-a9c2-b9a7516862d8 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:24.427841 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-327fb97e-ff55-4028-a9c2-b9a7516862d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:24.439056 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fb97e-ff55-4028-a9c2-b9a7516862d8 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:24.442624 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750b492-b281-4801-b049-198a80aa0fe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:24.443356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b492-b281-4801-b049-198a80aa0fe6 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:24.443839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b492-b281-4801-b049-198a80aa0fe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:24.444117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b492-b281-4801-b049-198a80aa0fe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:24.446045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c750b492-b281-4801-b049-198a80aa0fe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:24.514366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750b492-b281-4801-b049-198a80aa0fe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:24.516623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750b492-b281-4801-b049-198a80aa0fe6 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:24.530817 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1308/2599] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:57:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:57:25.603203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551e7cc-f51f-435d-b468-7109df8ae2ac service swift] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:25.604777 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4551e7cc-f51f-435d-b468-7109df8ae2ac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:25.619336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551e7cc-f51f-435d-b468-7109df8ae2ac service swift] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:25.624030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e535e56-4776-48c9-842e-eb820a28e860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:25.625099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e535e56-4776-48c9-842e-eb820a28e860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:25.625383 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e535e56-4776-48c9-842e-eb820a28e860 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:25.625659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e535e56-4776-48c9-842e-eb820a28e860 admin admin] PATH_INFO: `/v3/users/4812f8677a754dcd9e7a5607c643e8cb` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:25.626891 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e535e56-4776-48c9-842e-eb820a28e860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4812f8677a754dcd9e7a5607c643e8cb)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:25.628841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e535e56-4776-48c9-842e-eb820a28e860 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:25.675962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6e535e56-4776-48c9-842e-eb820a28e860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4812f8677a754dcd9e7a5607c643e8cb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:25.687369 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6e535e56-4776-48c9-842e-eb820a28e860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4812f8677a754dcd9e7a5607c643e8cb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:25.688039 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6e535e56-4776-48c9-842e-eb820a28e860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4812f8677a754dcd9e7a5607c643e8cb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:25.688642 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6e535e56-4776-48c9-842e-eb820a28e860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4812f8677a754dcd9e7a5607c643e8cb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:25.695581 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-6e535e56-4776-48c9-842e-eb820a28e860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4812f8677a754dcd9e7a5607c643e8cb'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:25.709358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1292/2600] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:57:25 2022] DELETE /identity/v3/users/4812f8677a754dcd9e7a5607c643e8cb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:57:25.979056 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750b492-b281-4801-b049-198a80aa0fe6 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:25.990605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750b492-b281-4801-b049-198a80aa0fe6 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:26.054971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:26.056137 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:26.056497 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:26.056839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] PATH_INFO: `/v3/projects/d37b24e8a85f49ce9d043e8a38b8748f` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:26.058151 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d37b24e8a85f49ce9d043e8a38b8748f)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:26.060311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:26.112883 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd37b24e8a85f49ce9d043e8a38b8748f', 'request_id': 'req-b6ad0f4b-894e-4ee9-a19a-501782a3468a'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.119238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd37b24e8a85f49ce9d043e8a38b8748f', 'request_id': 'req-b6ad0f4b-894e-4ee9-a19a-501782a3468a'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.128329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd37b24e8a85f49ce9d043e8a38b8748f', 'request_id': 'req-b6ad0f4b-894e-4ee9-a19a-501782a3468a'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.142679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] The token cache is being invalidate because project d37b24e8a85f49ce9d043e8a38b8748f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:57:26.143029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.143702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.145325 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1309/2601] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:57:25 2022] DELETE /identity/v3/projects/d37b24e8a85f49ce9d043e8a38b8748f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:57:26.149555 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e535e56-4776-48c9-842e-eb820a28e860 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:26.152078 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-6e535e56-4776-48c9-842e-eb820a28e860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:26.167976 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e535e56-4776-48c9-842e-eb820a28e860 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:26.228944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2efd5b-98ff-48f1-b9a0-7ac3499b0cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:26.229832 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2efd5b-98ff-48f1-b9a0-7ac3499b0cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:26.230119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2efd5b-98ff-48f1-b9a0-7ac3499b0cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:26.230399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2efd5b-98ff-48f1-b9a0-7ac3499b0cf8 admin admin] PATH_INFO: `/v3/users/df5b3cac540e4e6a88d131e757f36a4d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:26.231637 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2efd5b-98ff-48f1-b9a0-7ac3499b0cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df5b3cac540e4e6a88d131e757f36a4d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:26.233575 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2efd5b-98ff-48f1-b9a0-7ac3499b0cf8 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:26.283923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ff2efd5b-98ff-48f1-b9a0-7ac3499b0cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df5b3cac540e4e6a88d131e757f36a4d'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.295164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ff2efd5b-98ff-48f1-b9a0-7ac3499b0cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df5b3cac540e4e6a88d131e757f36a4d'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.295912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ff2efd5b-98ff-48f1-b9a0-7ac3499b0cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df5b3cac540e4e6a88d131e757f36a4d'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.296556 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ff2efd5b-98ff-48f1-b9a0-7ac3499b0cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df5b3cac540e4e6a88d131e757f36a4d'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.303237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ff2efd5b-98ff-48f1-b9a0-7ac3499b0cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df5b3cac540e4e6a88d131e757f36a4d'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.317572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1293/2602] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:57:26 2022] DELETE /identity/v3/users/df5b3cac540e4e6a88d131e757f36a4d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:57:26.487931 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:26.489676 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:26.503992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad0f4b-894e-4ee9-a19a-501782a3468a admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:26.567224 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:26.568328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:26.568975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:26.569175 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] PATH_INFO: `/v3/projects/370055efb92b4c5ea20501db02775ba8` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:26.570627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=370055efb92b4c5ea20501db02775ba8)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:26.573123 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:26.629777 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '370055efb92b4c5ea20501db02775ba8', 'request_id': 'req-d256f067-dfd0-4ebe-930c-948ee5616e4b'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.637450 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '370055efb92b4c5ea20501db02775ba8', 'request_id': 'req-d256f067-dfd0-4ebe-930c-948ee5616e4b'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.646011 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '370055efb92b4c5ea20501db02775ba8', 'request_id': 'req-d256f067-dfd0-4ebe-930c-948ee5616e4b'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.658594 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] The token cache is being invalidate because project 370055efb92b4c5ea20501db02775ba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:57:26.659048 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.659912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:26.662956 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1310/2603] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:57:26 2022] DELETE /identity/v3/projects/370055efb92b4c5ea20501db02775ba8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:57:27.173824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991e408-0bd2-4cc9-8af4-31094e346534 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:27.174332 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991e408-0bd2-4cc9-8af4-31094e346534 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:27.174594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991e408-0bd2-4cc9-8af4-31094e346534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:27.196982 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-7991e408-0bd2-4cc9-8af4-31094e346534 None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:57:27.198219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-7991e408-0bd2-4cc9-8af4-31094e346534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:27.260940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1294/2604] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:57:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:57:27.274785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:27.276801 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:27.351105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256f067-dfd0-4ebe-930c-948ee5616e4b admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:27.359087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba83165-c7d4-44fc-810f-a095ffffac84 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:27.359820 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba83165-c7d4-44fc-810f-a095ffffac84 service swift] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:27.360399 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba83165-c7d4-44fc-810f-a095ffffac84 service swift] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:27.360688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba83165-c7d4-44fc-810f-a095ffffac84 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:27.371317 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba83165-c7d4-44fc-810f-a095ffffac84 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:27.373071 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba83165-c7d4-44fc-810f-a095ffffac84 service swift] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:27.382742 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1311/2605] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:57:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:57:28.900462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991e408-0bd2-4cc9-8af4-31094e346534 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:28.902602 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-7991e408-0bd2-4cc9-8af4-31094e346534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:29.007975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991e408-0bd2-4cc9-8af4-31094e346534 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:29.015020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e677486-af4f-48d2-a4b6-83625dd48ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:29.016453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e677486-af4f-48d2-a4b6-83625dd48ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:29.016739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e677486-af4f-48d2-a4b6-83625dd48ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:29.017020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e677486-af4f-48d2-a4b6-83625dd48ea7 admin admin] PATH_INFO: `/v3/users/6b11a3de8b71482594a04ab7e53a94bd` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:29.018860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e677486-af4f-48d2-a4b6-83625dd48ea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b11a3de8b71482594a04ab7e53a94bd)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:29.021505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e677486-af4f-48d2-a4b6-83625dd48ea7 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:29.086048 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5e677486-af4f-48d2-a4b6-83625dd48ea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b11a3de8b71482594a04ab7e53a94bd'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:29.099119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5e677486-af4f-48d2-a4b6-83625dd48ea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b11a3de8b71482594a04ab7e53a94bd'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:29.100220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5e677486-af4f-48d2-a4b6-83625dd48ea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b11a3de8b71482594a04ab7e53a94bd'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:29.101103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5e677486-af4f-48d2-a4b6-83625dd48ea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b11a3de8b71482594a04ab7e53a94bd'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:29.108754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5e677486-af4f-48d2-a4b6-83625dd48ea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b11a3de8b71482594a04ab7e53a94bd'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:29.128514 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1295/2606] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:57:28 2022] DELETE /identity/v3/users/6b11a3de8b71482594a04ab7e53a94bd => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:57:29.342689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba83165-c7d4-44fc-810f-a095ffffac84 service swift] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:29.355581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba83165-c7d4-44fc-810f-a095ffffac84 service swift] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:29.450107 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:29.450722 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:29.451250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:29.451584 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c admin admin] PATH_INFO: `/v3/projects/428d047f511945e5bdf35eb8ed236507` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:29.463103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=428d047f511945e5bdf35eb8ed236507)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:29.468244 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:29.543768 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '428d047f511945e5bdf35eb8ed236507', 'request_id': 'req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:29.551525 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '428d047f511945e5bdf35eb8ed236507', 'request_id': 'req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:29.575145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e677486-af4f-48d2-a4b6-83625dd48ea7 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:29.577169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5e677486-af4f-48d2-a4b6-83625dd48ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:29.579555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '428d047f511945e5bdf35eb8ed236507', 'request_id': 'req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:29.597879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c admin admin] The token cache is being invalidate because project 428d047f511945e5bdf35eb8ed236507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:57:29.598101 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:29.599030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9ed93e93-01ec-4eb8-9b14-1441235bdb3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:57:29.601214 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1312/2607] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:57:29 2022] DELETE /identity/v3/projects/428d047f511945e5bdf35eb8ed236507 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:57:29.605365 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c6fc2-c5ad-4492-95ac-6459285bf9ea None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:29.605658 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c6fc2-c5ad-4492-95ac-6459285bf9ea None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:29.605939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c6fc2-c5ad-4492-95ac-6459285bf9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:29.636006 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-5c9c6fc2-c5ad-4492-95ac-6459285bf9ea None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:57:29.637310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-5c9c6fc2-c5ad-4492-95ac-6459285bf9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:29.696632 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e677486-af4f-48d2-a4b6-83625dd48ea7 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:29.723036 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1313/2608] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:57:29.730263 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39081ac9-ab32-4231-8af5-c1384a5f3080 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:29.730672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39081ac9-ab32-4231-8af5-c1384a5f3080 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:29.731076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39081ac9-ab32-4231-8af5-c1384a5f3080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:29.752500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-39081ac9-ab32-4231-8af5-c1384a5f3080 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:57:29.753962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-39081ac9-ab32-4231-8af5-c1384a5f3080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:29.762920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1314/2609] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:57:29.766983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39081ac9-ab32-4231-8af5-c1384a5f3080 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:29.778051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39081ac9-ab32-4231-8af5-c1384a5f3080 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:29.784122 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351979b4-f1b0-4376-b344-8ad7bfc66183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:29.784883 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351979b4-f1b0-4376-b344-8ad7bfc66183 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:29.785242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351979b4-f1b0-4376-b344-8ad7bfc66183 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:29.785589 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351979b4-f1b0-4376-b344-8ad7bfc66183 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:29.786387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351979b4-f1b0-4376-b344-8ad7bfc66183 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:57:29.786799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351979b4-f1b0-4376-b344-8ad7bfc66183 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:29.788476 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351979b4-f1b0-4376-b344-8ad7bfc66183 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:29.793158 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d042997-b871-46ee-b7bd-c431cfa88318 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:29.794028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d042997-b871-46ee-b7bd-c431cfa88318 service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:29.798444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d042997-b871-46ee-b7bd-c431cfa88318 service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:29.799343 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d042997-b871-46ee-b7bd-c431cfa88318 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:29.802226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-1d042997-b871-46ee-b7bd-c431cfa88318 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:29.802359 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1315/2610] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 21:57:29.806915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351979b4-f1b0-4376-b344-8ad7bfc66183 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:29.817454 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351979b4-f1b0-4376-b344-8ad7bfc66183 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:29.822084 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d042997-b871-46ee-b7bd-c431cfa88318 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:29.823766 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d042997-b871-46ee-b7bd-c431cfa88318 service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:29.823884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bbc09-0e39-4ba2-b984-648effb14371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:29.824558 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bbc09-0e39-4ba2-b984-648effb14371 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:29.824893 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bbc09-0e39-4ba2-b984-648effb14371 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:29.825242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bbc09-0e39-4ba2-b984-648effb14371 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:29.826001 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bbc09-0e39-4ba2-b984-648effb14371 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:29.828122 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bbc09-0e39-4ba2-b984-648effb14371 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:29.842336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1296/2611] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:57:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 21:57:29.873142 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d042997-b871-46ee-b7bd-c431cfa88318 service nova] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:29.873689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1316/2612] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:57:29 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:57:29.885657 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d042997-b871-46ee-b7bd-c431cfa88318 service nova] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:29.953945 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f36ab2-67ba-4966-8c32-690f6b759d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:29.954770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f36ab2-67ba-4966-8c32-690f6b759d7c admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:29.955134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f36ab2-67ba-4966-8c32-690f6b759d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:29.955490 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f36ab2-67ba-4966-8c32-690f6b759d7c admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:29.956256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f36ab2-67ba-4966-8c32-690f6b759d7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:29.958267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f36ab2-67ba-4966-8c32-690f6b759d7c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:29.990406 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1297/2613] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:57:29 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:57:29.991836 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bbc09-0e39-4ba2-b984-648effb14371 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:30.002902 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bbc09-0e39-4ba2-b984-648effb14371 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:30.008259 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faadd2d-9691-47f7-b460-d3049c4926f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:30.009051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faadd2d-9691-47f7-b460-d3049c4926f1 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:30.009356 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faadd2d-9691-47f7-b460-d3049c4926f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:30.009744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faadd2d-9691-47f7-b460-d3049c4926f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:30.010563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faadd2d-9691-47f7-b460-d3049c4926f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:30.012334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faadd2d-9691-47f7-b460-d3049c4926f1 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:30.023951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1317/2614] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:57:30.028898 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f36ab2-67ba-4966-8c32-690f6b759d7c admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:30.042542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f36ab2-67ba-4966-8c32-690f6b759d7c admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:30.047966 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae668e-5ef0-4d75-8cec-1cf772c72754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:30.048627 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae668e-5ef0-4d75-8cec-1cf772c72754 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:30.048915 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae668e-5ef0-4d75-8cec-1cf772c72754 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:30.049194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae668e-5ef0-4d75-8cec-1cf772c72754 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:30.049930 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae668e-5ef0-4d75-8cec-1cf772c72754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:30.051662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae668e-5ef0-4d75-8cec-1cf772c72754 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:30.063541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1298/2615] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 21:57:30.067487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faadd2d-9691-47f7-b460-d3049c4926f1 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:30.080202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faadd2d-9691-47f7-b460-d3049c4926f1 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:30.086009 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f13617-0075-4d0e-b9bf-f86dd7719ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:30.087212 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f13617-0075-4d0e-b9bf-f86dd7719ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:30.087709 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f13617-0075-4d0e-b9bf-f86dd7719ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:30.088419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f13617-0075-4d0e-b9bf-f86dd7719ac0 admin admin] PATH_INFO: `/v3/projects/a04ed773a4b44bf5b881acbdf51a8463/users/ce6d80446f54487387c7f58eeb3c83c1/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:30.108900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f13617-0075-4d0e-b9bf-f86dd7719ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a04ed773a4b44bf5b881acbdf51a8463, user_id=ce6d80446f54487387c7f58eeb3c83c1, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:30.112583 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f13617-0075-4d0e-b9bf-f86dd7719ac0 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:30.130896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1318/2616] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:57:30 2022] PUT /identity/v3/projects/a04ed773a4b44bf5b881acbdf51a8463/users/ce6d80446f54487387c7f58eeb3c83c1/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 21:57:30.132568 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ca755-83c3-46cf-91bf-de1ea40020b8 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:30.132880 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ca755-83c3-46cf-91bf-de1ea40020b8 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:30.133160 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ca755-83c3-46cf-91bf-de1ea40020b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:30.156722 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-380ca755-83c3-46cf-91bf-de1ea40020b8 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:57:30.157961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-380ca755-83c3-46cf-91bf-de1ea40020b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:30.225066 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1299/2617] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:57:30.236019 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f13617-0075-4d0e-b9bf-f86dd7719ac0 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:30.237700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-14f13617-0075-4d0e-b9bf-f86dd7719ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:30.306312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f13617-0075-4d0e-b9bf-f86dd7719ac0 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:30.310478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1f267-5053-4a74-873f-4a5e518ba915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:30.312182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1f267-5053-4a74-873f-4a5e518ba915 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:30.312182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1f267-5053-4a74-873f-4a5e518ba915 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:30.312182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1f267-5053-4a74-873f-4a5e518ba915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:30.323000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c1f267-5053-4a74-873f-4a5e518ba915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:30.324460 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c1f267-5053-4a74-873f-4a5e518ba915 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:30.334995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1319/2618] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:57:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:57:35.674984 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7864f-a1eb-49e8-9f74-0d5845563eea None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:35.675542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7864f-a1eb-49e8-9f74-0d5845563eea None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:35.675542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7864f-a1eb-49e8-9f74-0d5845563eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:35.722781 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-91c7864f-a1eb-49e8-9f74-0d5845563eea None None] MFA Rules not processed for user `ce6d80446f54487387c7f58eeb3c83c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:57:35.725130 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-91c7864f-a1eb-49e8-9f74-0d5845563eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:35.813671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1300/2619] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:57:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:57:35.828859 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1f267-5053-4a74-873f-4a5e518ba915 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:35.830453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b7c1f267-5053-4a74-873f-4a5e518ba915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:35.950078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1f267-5053-4a74-873f-4a5e518ba915 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:35.973788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3da7b-35df-4d1e-a739-d5ad48de942f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:35.974589 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3da7b-35df-4d1e-a739-d5ad48de942f service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:35.975016 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3da7b-35df-4d1e-a739-d5ad48de942f service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:35.975400 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3da7b-35df-4d1e-a739-d5ad48de942f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:35.993045 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3da7b-35df-4d1e-a739-d5ad48de942f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:35.995324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3da7b-35df-4d1e-a739-d5ad48de942f service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:36.007952 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1320/2620] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:57:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:57:37.706502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa2058-3fd9-414f-9089-46b85d21ff50 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:37.707074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa2058-3fd9-414f-9089-46b85d21ff50 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:37.707212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa2058-3fd9-414f-9089-46b85d21ff50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:37.730485 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-56fa2058-3fd9-414f-9089-46b85d21ff50 None None] MFA Rules not processed for user `ce6d80446f54487387c7f58eeb3c83c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:57:37.731709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-56fa2058-3fd9-414f-9089-46b85d21ff50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:37.749923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1301/2621] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:57:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:57:38.136390 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3da7b-35df-4d1e-a739-d5ad48de942f service nova] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:38.139745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-21d3da7b-35df-4d1e-a739-d5ad48de942f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:38.158417 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3da7b-35df-4d1e-a739-d5ad48de942f service nova] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:38.173346 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615e3b1-c4ff-4299-9cfb-1990d2da889b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:38.174102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615e3b1-c4ff-4299-9cfb-1990d2da889b service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:38.174458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615e3b1-c4ff-4299-9cfb-1990d2da889b service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:38.174869 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615e3b1-c4ff-4299-9cfb-1990d2da889b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:38.179422 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-4615e3b1-c4ff-4299-9cfb-1990d2da889b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:38.204148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615e3b1-c4ff-4299-9cfb-1990d2da889b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:38.204881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615e3b1-c4ff-4299-9cfb-1990d2da889b service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:38.213769 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1321/2622] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:57:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:57:38.799161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa2058-3fd9-414f-9089-46b85d21ff50 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:38.801331 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-56fa2058-3fd9-414f-9089-46b85d21ff50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:38.941073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa2058-3fd9-414f-9089-46b85d21ff50 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:38.950099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e667cc-1829-4887-8e49-09c11dc91c99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8124ab8abcc640c5a48b3e0216afa48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:38.951003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e667cc-1829-4887-8e49-09c11dc91c99 service placement] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:38.952049 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e667cc-1829-4887-8e49-09c11dc91c99 service placement] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:38.953044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e667cc-1829-4887-8e49-09c11dc91c99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:38.955463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b3e667cc-1829-4887-8e49-09c11dc91c99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:38.975656 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e667cc-1829-4887-8e49-09c11dc91c99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:38.977513 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e667cc-1829-4887-8e49-09c11dc91c99 service placement] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:39.002815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1302/2623] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:57:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:57:44.615187 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8a21b-717f-473d-863c-8bd9b05dfa5c None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:44.615818 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8a21b-717f-473d-863c-8bd9b05dfa5c None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:44.615941 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8a21b-717f-473d-863c-8bd9b05dfa5c None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:44.617501 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1322/2624] 10.222.0.35 () {54 vars in 921 bytes} [Fri May 13 21:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:57:44.623589 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca3e94-0997-4a1f-a32a-5b698def204b None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:44.624161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca3e94-0997-4a1f-a32a-5b698def204b None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:44.625025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca3e94-0997-4a1f-a32a-5b698def204b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:44.661824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-0fca3e94-0997-4a1f-a32a-5b698def204b None None] MFA Rules not processed for user `316ef43c6c304bd3a27e085bf4da8a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:57:44.663127 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0fca3e94-0997-4a1f-a32a-5b698def204b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:44.673125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1303/2625] 10.222.0.35 () {58 vars in 1018 bytes} [Fri May 13 21:57:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:57:44.684901 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8a21b-717f-473d-863c-8bd9b05dfa5c None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:44.687283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-28e8a21b-717f-473d-863c-8bd9b05dfa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:44.751885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8a21b-717f-473d-863c-8bd9b05dfa5c None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:44.756048 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10fb04-3249-4315-9424-4cfaa8aa05bf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad57b046f3a43978d1c98bcc1a76d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:44.756699 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10fb04-3249-4315-9424-4cfaa8aa05bf service glance] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:44.757247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10fb04-3249-4315-9424-4cfaa8aa05bf service glance] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:44.757530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10fb04-3249-4315-9424-4cfaa8aa05bf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:44.768615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10fb04-3249-4315-9424-4cfaa8aa05bf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:44.770078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10fb04-3249-4315-9424-4cfaa8aa05bf service glance] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:44.781315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1323/2626] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:57:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:57:44.851169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a8149-7161-4706-929c-ab2c70fece8a None None] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:44.851531 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a8149-7161-4706-929c-ab2c70fece8a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:44.851910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a8149-7161-4706-929c-ab2c70fece8a None None] PATH_INFO: `/` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:44.853567 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1304/2627] 10.222.0.35 () {54 vars in 921 bytes} [Fri May 13 21:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:57:44.858187 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb142cf-be0e-47d2-8d67-452926da9355 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:44.858638 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb142cf-be0e-47d2-8d67-452926da9355 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:44.859041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb142cf-be0e-47d2-8d67-452926da9355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:44.879105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-aeb142cf-be0e-47d2-8d67-452926da9355 None None] MFA Rules not processed for user `c4df058606c448a1bb757020e869ac33`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:57:44.880401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-aeb142cf-be0e-47d2-8d67-452926da9355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:44.886987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1324/2628] 10.222.0.35 () {58 vars in 1018 bytes} [Fri May 13 21:57:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:57:44.899513 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a8149-7161-4706-929c-ab2c70fece8a None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:44.910198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a8149-7161-4706-929c-ab2c70fece8a None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:44.913904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f4c89-7199-4aa7-a6b4-6d9eec5178ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:44.914558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f4c89-7199-4aa7-a6b4-6d9eec5178ff service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:44.914898 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f4c89-7199-4aa7-a6b4-6d9eec5178ff service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:44.915337 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f4c89-7199-4aa7-a6b4-6d9eec5178ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:44.923085 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f4c89-7199-4aa7-a6b4-6d9eec5178ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:44.924645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f4c89-7199-4aa7-a6b4-6d9eec5178ff service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:44.932611 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1305/2629] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:57:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:57:45.020058 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60525f0f-c8cc-4ae0-b1b7-d9e847439703 None None] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:45.020349 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60525f0f-c8cc-4ae0-b1b7-d9e847439703 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:45.020626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60525f0f-c8cc-4ae0-b1b7-d9e847439703 None None] PATH_INFO: `/` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:45.022005 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1325/2630] 10.222.0.35 () {54 vars in 921 bytes} [Fri May 13 21:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 21:57:45.027341 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0442a28-eceb-4430-b0cd-ffa9b687e8e1 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:45.027700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0442a28-eceb-4430-b0cd-ffa9b687e8e1 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:45.028046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0442a28-eceb-4430-b0cd-ffa9b687e8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:45.045775 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-c0442a28-eceb-4430-b0cd-ffa9b687e8e1 None None] MFA Rules not processed for user `316ef43c6c304bd3a27e085bf4da8a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:57:45.052315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c0442a28-eceb-4430-b0cd-ffa9b687e8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:45.063042 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1306/2631] 10.222.0.35 () {58 vars in 1018 bytes} [Fri May 13 21:57:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:57:45.074309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60525f0f-c8cc-4ae0-b1b7-d9e847439703 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:45.087720 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60525f0f-c8cc-4ae0-b1b7-d9e847439703 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:45.091832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dbe7b-2dc7-4214-affe-968cfaaac8ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:45.092515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dbe7b-2dc7-4214-affe-968cfaaac8ab service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:45.092802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dbe7b-2dc7-4214-affe-968cfaaac8ab service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:45.093080 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dbe7b-2dc7-4214-affe-968cfaaac8ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:45.103689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6dbe7b-2dc7-4214-affe-968cfaaac8ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:45.105128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6dbe7b-2dc7-4214-affe-968cfaaac8ab service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:45.116306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1326/2632] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:57:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:57:46.799631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23281f-1b66-4774-954b-aee83551cd6d None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:46.800114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23281f-1b66-4774-954b-aee83551cd6d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:46.800221 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23281f-1b66-4774-954b-aee83551cd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:46.817153 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-1b23281f-1b66-4774-954b-aee83551cd6d None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:57:46.818331 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-1b23281f-1b66-4774-954b-aee83551cd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:46.871706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1307/2633] 10.222.0.35 () {58 vars in 1009 bytes} [Fri May 13 21:57:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:57:46.883293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dbe7b-2dc7-4214-affe-968cfaaac8ab service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:57:46.884667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-1d6dbe7b-2dc7-4214-affe-968cfaaac8ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:57:46.948933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dbe7b-2dc7-4214-affe-968cfaaac8ab service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:57:46.954396 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebaab4-be88-472d-98ab-8147f49e0641 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:57:46.955053 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebaab4-be88-472d-98ab-8147f49e0641 service swift] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:57:46.955335 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebaab4-be88-472d-98ab-8147f49e0641 service swift] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:57:46.955606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebaab4-be88-472d-98ab-8147f49e0641 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:57:46.963244 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebaab4-be88-472d-98ab-8147f49e0641 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:57:46.964813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebaab4-be88-472d-98ab-8147f49e0641 service swift] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:57:46.973005 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1327/2634] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:57:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 13 21:58:01.081774 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef79845-6f1c-4fd1-a401-23b3204d0c95 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:01.082266 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef79845-6f1c-4fd1-a401-23b3204d0c95 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:01.082367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef79845-6f1c-4fd1-a401-23b3204d0c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:01.100198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-bef79845-6f1c-4fd1-a401-23b3204d0c95 None None] MFA Rules not processed for user `a079ae1557d04831b73c03c96e2c7ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:58:01.101456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-bef79845-6f1c-4fd1-a401-23b3204d0c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:01.108439 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1308/2635] 10.222.0.35 () {58 vars in 967 bytes} [Fri May 13 21:58:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:58:01.120629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebaab4-be88-472d-98ab-8147f49e0641 service swift] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:58:01.131007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebaab4-be88-472d-98ab-8147f49e0641 service swift] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:58:01.135896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce5f29-4b78-49b8-8237-da3d8a7fb8f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f9e07ed0a084c82b946aaf585de16fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:58:01.136510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce5f29-4b78-49b8-8237-da3d8a7fb8f4 service swift] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:01.136787 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce5f29-4b78-49b8-8237-da3d8a7fb8f4 service swift] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:01.137059 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce5f29-4b78-49b8-8237-da3d8a7fb8f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:01.145314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce5f29-4b78-49b8-8237-da3d8a7fb8f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:58:01.146914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce5f29-4b78-49b8-8237-da3d8a7fb8f4 service swift] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:58:01.154999 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1328/2636] 10.222.0.35 () {60 vars in 1384 bytes} [Fri May 13 21:58:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 21:58:28.694035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef79845-6f1c-4fd1-a401-23b3204d0c95 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:58:28.695506 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-bef79845-6f1c-4fd1-a401-23b3204d0c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:28.749702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef79845-6f1c-4fd1-a401-23b3204d0c95 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:58:28.753897 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ac602-42c6-48d0-8e47-eb42f18cd71c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:58:28.754540 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ac602-42c6-48d0-8e47-eb42f18cd71c service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:28.754872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ac602-42c6-48d0-8e47-eb42f18cd71c service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:28.755155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ac602-42c6-48d0-8e47-eb42f18cd71c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:28.756913 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-aa0ac602-42c6-48d0-8e47-eb42f18cd71c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:28.811583 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ac602-42c6-48d0-8e47-eb42f18cd71c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:58:28.813606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ac602-42c6-48d0-8e47-eb42f18cd71c service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:58:28.822341 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1309/2637] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:58:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:58:36.626740 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0540e83-7707-4c90-be58-09991116bea4 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:36.627362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0540e83-7707-4c90-be58-09991116bea4 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:36.627362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0540e83-7707-4c90-be58-09991116bea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:36.646238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a0540e83-7707-4c90-be58-09991116bea4 None None] MFA Rules not processed for user `c3a0de9ca12746be893b97d3cd11b826`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:58:36.647487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a0540e83-7707-4c90-be58-09991116bea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:36.654996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1329/2638] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:58:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:58:36.964240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ac602-42c6-48d0-8e47-eb42f18cd71c service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:58:36.973411 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ac602-42c6-48d0-8e47-eb42f18cd71c service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:58:36.976963 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70920071-4752-4efc-8bde-2256bc769463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:58:36.977596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70920071-4752-4efc-8bde-2256bc769463 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:36.977882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70920071-4752-4efc-8bde-2256bc769463 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:36.978159 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70920071-4752-4efc-8bde-2256bc769463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:36.979922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-70920071-4752-4efc-8bde-2256bc769463 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:36.989920 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70920071-4752-4efc-8bde-2256bc769463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:58:36.991373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70920071-4752-4efc-8bde-2256bc769463 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:58:36.999179 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1310/2639] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:58:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:58:46.482557 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da2d37-ba36-4e45-97ec-76c66902bce3 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:46.483061 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da2d37-ba36-4e45-97ec-76c66902bce3 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:46.483182 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da2d37-ba36-4e45-97ec-76c66902bce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:46.501063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-c0da2d37-ba36-4e45-97ec-76c66902bce3 None None] MFA Rules not processed for user `ce6d80446f54487387c7f58eeb3c83c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:58:46.502256 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c0da2d37-ba36-4e45-97ec-76c66902bce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:46.510877 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1330/2640] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:58:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:58:48.872709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70920071-4752-4efc-8bde-2256bc769463 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:58:48.882606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70920071-4752-4efc-8bde-2256bc769463 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:58:48.888237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db762c-d7c9-4216-8f4b-fd6b318663da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:58:48.888901 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db762c-d7c9-4216-8f4b-fd6b318663da service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:48.889270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db762c-d7c9-4216-8f4b-fd6b318663da service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:48.889675 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db762c-d7c9-4216-8f4b-fd6b318663da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:48.892157 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-44db762c-d7c9-4216-8f4b-fd6b318663da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:48.905128 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db762c-d7c9-4216-8f4b-fd6b318663da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:58:48.908453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db762c-d7c9-4216-8f4b-fd6b318663da service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:58:48.922967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1311/2641] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:58:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:58:52.329447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da2d37-ba36-4e45-97ec-76c66902bce3 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:58:52.331529 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c0da2d37-ba36-4e45-97ec-76c66902bce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:52.348159 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da2d37-ba36-4e45-97ec-76c66902bce3 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:58:52.352731 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613cacb-0e3f-446c-9268-b42788381022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:58:52.353590 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613cacb-0e3f-446c-9268-b42788381022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:52.353875 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613cacb-0e3f-446c-9268-b42788381022 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:52.354152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613cacb-0e3f-446c-9268-b42788381022 admin admin] PATH_INFO: `/v3/users/0a1425d092844a2e9b8a917dc9981b32` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:52.355349 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613cacb-0e3f-446c-9268-b42788381022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a1425d092844a2e9b8a917dc9981b32)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:58:52.357289 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613cacb-0e3f-446c-9268-b42788381022 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:58:52.404269 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3613cacb-0e3f-446c-9268-b42788381022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a1425d092844a2e9b8a917dc9981b32'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:58:52.416465 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3613cacb-0e3f-446c-9268-b42788381022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a1425d092844a2e9b8a917dc9981b32'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:58:52.417105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3613cacb-0e3f-446c-9268-b42788381022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a1425d092844a2e9b8a917dc9981b32'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:58:52.417716 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3613cacb-0e3f-446c-9268-b42788381022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a1425d092844a2e9b8a917dc9981b32'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:58:52.424482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-3613cacb-0e3f-446c-9268-b42788381022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a1425d092844a2e9b8a917dc9981b32'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:58:52.437447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1331/2642] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 21:58:52 2022] DELETE /identity/v3/users/0a1425d092844a2e9b8a917dc9981b32 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:58:52.873815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db762c-d7c9-4216-8f4b-fd6b318663da service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:58:52.889653 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db762c-d7c9-4216-8f4b-fd6b318663da service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:58:52.951964 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167640e-6aad-484f-90a9-821ea0a34082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:58:52.952828 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167640e-6aad-484f-90a9-821ea0a34082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:52.953161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167640e-6aad-484f-90a9-821ea0a34082 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:52.953949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167640e-6aad-484f-90a9-821ea0a34082 admin admin] PATH_INFO: `/v3/projects/6a12e6fa2119499b93806f036eb55a0c` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:52.955180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167640e-6aad-484f-90a9-821ea0a34082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a12e6fa2119499b93806f036eb55a0c)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:58:52.957088 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167640e-6aad-484f-90a9-821ea0a34082 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:58:53.011674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3167640e-6aad-484f-90a9-821ea0a34082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a12e6fa2119499b93806f036eb55a0c', 'request_id': 'req-3167640e-6aad-484f-90a9-821ea0a34082'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:58:53.018615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3167640e-6aad-484f-90a9-821ea0a34082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a12e6fa2119499b93806f036eb55a0c', 'request_id': 'req-3167640e-6aad-484f-90a9-821ea0a34082'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:58:53.026614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3167640e-6aad-484f-90a9-821ea0a34082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a12e6fa2119499b93806f036eb55a0c', 'request_id': 'req-3167640e-6aad-484f-90a9-821ea0a34082'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:58:53.037477 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3167640e-6aad-484f-90a9-821ea0a34082 admin admin] The token cache is being invalidate because project 6a12e6fa2119499b93806f036eb55a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 21:58:53.037771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3167640e-6aad-484f-90a9-821ea0a34082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:58:53.038419 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-3167640e-6aad-484f-90a9-821ea0a34082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 21:58:53.040047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1312/2643] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 21:58:52 2022] DELETE /identity/v3/projects/6a12e6fa2119499b93806f036eb55a0c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:58:53.044879 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394c9cb-c650-4c80-958a-ee92f6fd4636 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:53.045171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394c9cb-c650-4c80-958a-ee92f6fd4636 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:53.045451 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394c9cb-c650-4c80-958a-ee92f6fd4636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:53.064518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-c394c9cb-c650-4c80-958a-ee92f6fd4636 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:58:53.066039 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-c394c9cb-c650-4c80-958a-ee92f6fd4636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:53.131225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1332/2644] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:58:53.139232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e44f55-f366-4805-b08a-58ae802908d1 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:53.139594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e44f55-f366-4805-b08a-58ae802908d1 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:53.139993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e44f55-f366-4805-b08a-58ae802908d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:53.159412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-63e44f55-f366-4805-b08a-58ae802908d1 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:58:53.160697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-63e44f55-f366-4805-b08a-58ae802908d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:53.169710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1313/2645] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:58:53.174381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394c9cb-c650-4c80-958a-ee92f6fd4636 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:58:53.183926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394c9cb-c650-4c80-958a-ee92f6fd4636 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:58:53.196485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275e342-b86d-468f-8365-052c8d9324bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:58:53.197251 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275e342-b86d-468f-8365-052c8d9324bb admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:53.197448 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275e342-b86d-468f-8365-052c8d9324bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:53.197719 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275e342-b86d-468f-8365-052c8d9324bb admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:53.198445 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275e342-b86d-468f-8365-052c8d9324bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 21:58:53.198774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275e342-b86d-468f-8365-052c8d9324bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:58:53.200384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275e342-b86d-468f-8365-052c8d9324bb admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:58:53.212759 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1333/2646] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 21:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 21:58:53.217008 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e44f55-f366-4805-b08a-58ae802908d1 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:58:53.227545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e44f55-f366-4805-b08a-58ae802908d1 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:58:53.233182 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fae75-2dd3-4dd5-8cf1-833508ede5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:58:53.234098 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fae75-2dd3-4dd5-8cf1-833508ede5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:53.234937 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fae75-2dd3-4dd5-8cf1-833508ede5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:53.235220 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fae75-2dd3-4dd5-8cf1-833508ede5a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:53.236015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fae75-2dd3-4dd5-8cf1-833508ede5a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:58:53.237922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fae75-2dd3-4dd5-8cf1-833508ede5a9 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:58:53.273595 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1314/2647] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 21:58:53 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 21:58:53.277845 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275e342-b86d-468f-8365-052c8d9324bb admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:58:53.287953 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275e342-b86d-468f-8365-052c8d9324bb admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:58:53.344833 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6737c2-25cf-4b10-a1df-cf1d72337e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:58:53.346120 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6737c2-25cf-4b10-a1df-cf1d72337e94 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:53.346437 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6737c2-25cf-4b10-a1df-cf1d72337e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:53.346729 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6737c2-25cf-4b10-a1df-cf1d72337e94 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:53.347458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6737c2-25cf-4b10-a1df-cf1d72337e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:58:53.349448 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6737c2-25cf-4b10-a1df-cf1d72337e94 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:58:53.370168 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1334/2648] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 21:58:53 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 21:58:53.374169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fae75-2dd3-4dd5-8cf1-833508ede5a9 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:58:53.384962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fae75-2dd3-4dd5-8cf1-833508ede5a9 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:58:53.389816 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe7df5-551e-4973-9e32-405f779b3283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:58:53.390486 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe7df5-551e-4973-9e32-405f779b3283 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:53.390851 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe7df5-551e-4973-9e32-405f779b3283 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:53.391134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe7df5-551e-4973-9e32-405f779b3283 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:53.391870 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe7df5-551e-4973-9e32-405f779b3283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:58:53.393474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe7df5-551e-4973-9e32-405f779b3283 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:58:53.403384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1315/2649] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:58:53.407898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6737c2-25cf-4b10-a1df-cf1d72337e94 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:58:53.418195 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6737c2-25cf-4b10-a1df-cf1d72337e94 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:58:53.423394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6c261-f3d4-4d33-ac8d-d844ae24d306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:58:53.424046 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6c261-f3d4-4d33-ac8d-d844ae24d306 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:53.424330 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6c261-f3d4-4d33-ac8d-d844ae24d306 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:53.424625 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6c261-f3d4-4d33-ac8d-d844ae24d306 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:53.425357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6c261-f3d4-4d33-ac8d-d844ae24d306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:58:53.426987 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6c261-f3d4-4d33-ac8d-d844ae24d306 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:58:53.437060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1335/2650] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 21:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 21:58:53.441109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe7df5-551e-4973-9e32-405f779b3283 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:58:53.451123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe7df5-551e-4973-9e32-405f779b3283 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:58:53.455689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95303fa8-59c5-433a-975e-9cf6e7aa95b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:58:53.456553 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95303fa8-59c5-433a-975e-9cf6e7aa95b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:53.456837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95303fa8-59c5-433a-975e-9cf6e7aa95b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:53.457113 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95303fa8-59c5-433a-975e-9cf6e7aa95b2 admin admin] PATH_INFO: `/v3/projects/9e5d643885114a45b305ac13bef897d3/users/dddbe23734ee4cee866d58e3314007ea/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:53.472673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95303fa8-59c5-433a-975e-9cf6e7aa95b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5d643885114a45b305ac13bef897d3, user_id=dddbe23734ee4cee866d58e3314007ea, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:58:53.475317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95303fa8-59c5-433a-975e-9cf6e7aa95b2 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:58:53.488405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1316/2651] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 21:58:53 2022] PUT /identity/v3/projects/9e5d643885114a45b305ac13bef897d3/users/dddbe23734ee4cee866d58e3314007ea/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 21:58:53.493385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102952e-a108-4c3c-90c5-e8352a5561c7 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:53.493702 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102952e-a108-4c3c-90c5-e8352a5561c7 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:53.493983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102952e-a108-4c3c-90c5-e8352a5561c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:53.511373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-6102952e-a108-4c3c-90c5-e8352a5561c7 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:58:53.512557 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-6102952e-a108-4c3c-90c5-e8352a5561c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:53.569819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1336/2652] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:58:53.579451 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95303fa8-59c5-433a-975e-9cf6e7aa95b2 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:58:53.581161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-95303fa8-59c5-433a-975e-9cf6e7aa95b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:53.635652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95303fa8-59c5-433a-975e-9cf6e7aa95b2 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:58:53.639376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85799f-e6a2-4287-8e1d-cbdb8b963682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:58:53.640073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85799f-e6a2-4287-8e1d-cbdb8b963682 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:53.640364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85799f-e6a2-4287-8e1d-cbdb8b963682 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:53.640645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85799f-e6a2-4287-8e1d-cbdb8b963682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:53.649516 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85799f-e6a2-4287-8e1d-cbdb8b963682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:58:53.650949 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85799f-e6a2-4287-8e1d-cbdb8b963682 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:58:53.660521 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1317/2653] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:58:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:58:57.896099 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b733ce0-b613-4540-a80e-12d5ded47033 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:57.896976 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b733ce0-b613-4540-a80e-12d5ded47033 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:57.897500 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b733ce0-b613-4540-a80e-12d5ded47033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:57.927918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-0b733ce0-b613-4540-a80e-12d5ded47033 None None] MFA Rules not processed for user `dddbe23734ee4cee866d58e3314007ea`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:58:57.929448 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-0b733ce0-b613-4540-a80e-12d5ded47033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:58.009596 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1337/2654] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:58:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 21:58:58.023710 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85799f-e6a2-4287-8e1d-cbdb8b963682 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:58:58.025304 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2a85799f-e6a2-4287-8e1d-cbdb8b963682 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:58.107574 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85799f-e6a2-4287-8e1d-cbdb8b963682 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:58:58.114537 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07bd7e-046f-4942-90eb-d43678746de1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:58:58.115815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07bd7e-046f-4942-90eb-d43678746de1 service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:58.116283 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07bd7e-046f-4942-90eb-d43678746de1 service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:58.116768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07bd7e-046f-4942-90eb-d43678746de1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:58.125093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07bd7e-046f-4942-90eb-d43678746de1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:58:58.126722 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07bd7e-046f-4942-90eb-d43678746de1 service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:58:58.136400 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1318/2655] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:58:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:58:59.886821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef635f-5572-481f-bd3d-4a0bf1d0d0f1 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:58:59.887340 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef635f-5572-481f-bd3d-4a0bf1d0d0f1 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:58:59.887450 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef635f-5572-481f-bd3d-4a0bf1d0d0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:58:59.904740 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-51ef635f-5572-481f-bd3d-4a0bf1d0d0f1 None None] MFA Rules not processed for user `dddbe23734ee4cee866d58e3314007ea`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:58:59.905928 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-51ef635f-5572-481f-bd3d-4a0bf1d0d0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:58:59.913015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1338/2656] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:58:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 21:59:01.891643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07bd7e-046f-4942-90eb-d43678746de1 service nova] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 21:59:01.893121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-6b07bd7e-046f-4942-90eb-d43678746de1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:59:01.926975 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07bd7e-046f-4942-90eb-d43678746de1 service nova] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 21:59:01.943612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec82613-82e2-483d-a7ab-b33000d81ee7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 21:59:01.945106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec82613-82e2-483d-a7ab-b33000d81ee7 service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:59:01.945576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec82613-82e2-483d-a7ab-b33000d81ee7 service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:59:01.946028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec82613-82e2-483d-a7ab-b33000d81ee7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:59:01.948959 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-eec82613-82e2-483d-a7ab-b33000d81ee7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:59:01.964804 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec82613-82e2-483d-a7ab-b33000d81ee7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 21:59:01.967309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec82613-82e2-483d-a7ab-b33000d81ee7 service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 21:59:01.984490 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1319/2657] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 21:59:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 21:59:49.477447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff836079-a7d5-41f0-baca-dfb051f8875b None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 21:59:49.477981 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff836079-a7d5-41f0-baca-dfb051f8875b None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 21:59:49.478107 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff836079-a7d5-41f0-baca-dfb051f8875b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 21:59:49.497807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-ff836079-a7d5-41f0-baca-dfb051f8875b None None] MFA Rules not processed for user `ce6d80446f54487387c7f58eeb3c83c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 21:59:49.499643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ff836079-a7d5-41f0-baca-dfb051f8875b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 21:59:49.556632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1339/2658] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 21:59:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:00:39.569208 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec82613-82e2-483d-a7ab-b33000d81ee7 service nova] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:00:39.570781 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-eec82613-82e2-483d-a7ab-b33000d81ee7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:00:39.642373 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec82613-82e2-483d-a7ab-b33000d81ee7 service nova] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:00:39.648304 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dfa106-dddc-4f57-989e-ccd1eeafdbb8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8124ab8abcc640c5a48b3e0216afa48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:00:39.649079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfa106-dddc-4f57-989e-ccd1eeafdbb8 service placement] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:00:39.649421 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfa106-dddc-4f57-989e-ccd1eeafdbb8 service placement] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:00:39.649770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfa106-dddc-4f57-989e-ccd1eeafdbb8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:00:39.651645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c1dfa106-dddc-4f57-989e-ccd1eeafdbb8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:00:39.664099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dfa106-dddc-4f57-989e-ccd1eeafdbb8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:00:39.666018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dfa106-dddc-4f57-989e-ccd1eeafdbb8 service placement] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:00:39.677287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1320/2659] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:00:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:00:51.275591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff836079-a7d5-41f0-baca-dfb051f8875b None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:00:51.285237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff836079-a7d5-41f0-baca-dfb051f8875b None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:00:51.288871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f7a87-13a5-4d6d-b4f6-4161176ad17b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:00:51.289789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f7a87-13a5-4d6d-b4f6-4161176ad17b service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:00:51.290068 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f7a87-13a5-4d6d-b4f6-4161176ad17b service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:00:51.290502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f7a87-13a5-4d6d-b4f6-4161176ad17b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:00:51.292528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-2c5f7a87-13a5-4d6d-b4f6-4161176ad17b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:00:51.305803 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f7a87-13a5-4d6d-b4f6-4161176ad17b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:00:51.307416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f7a87-13a5-4d6d-b4f6-4161176ad17b service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:00:51.319204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1340/2660] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:00:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:00:56.057266 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a4089-a7ba-4276-9d2c-6464b84bcd81 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:00:56.057768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a4089-a7ba-4276-9d2c-6464b84bcd81 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:00:56.058002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a4089-a7ba-4276-9d2c-6464b84bcd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:00:56.079793 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-af6a4089-a7ba-4276-9d2c-6464b84bcd81 None None] MFA Rules not processed for user `ce6d80446f54487387c7f58eeb3c83c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:00:56.081734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-af6a4089-a7ba-4276-9d2c-6464b84bcd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:00:56.089104 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1321/2661] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:00:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:00:58.833409 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f7a87-13a5-4d6d-b4f6-4161176ad17b service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:00:58.835153 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-2c5f7a87-13a5-4d6d-b4f6-4161176ad17b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:00:58.850477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f7a87-13a5-4d6d-b4f6-4161176ad17b service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:00:58.856511 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4adf1b-9120-4c29-9d56-2b0b4b8bb0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:00:58.857701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4adf1b-9120-4c29-9d56-2b0b4b8bb0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:00:58.858210 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4adf1b-9120-4c29-9d56-2b0b4b8bb0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:00:58.858686 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4adf1b-9120-4c29-9d56-2b0b4b8bb0c5 admin admin] PATH_INFO: `/v3/users/c3a0de9ca12746be893b97d3cd11b826` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:00:58.860384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4adf1b-9120-4c29-9d56-2b0b4b8bb0c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a0de9ca12746be893b97d3cd11b826)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:00:58.862359 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4adf1b-9120-4c29-9d56-2b0b4b8bb0c5 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:00:58.915199 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-5c4adf1b-9120-4c29-9d56-2b0b4b8bb0c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a0de9ca12746be893b97d3cd11b826'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:58.928352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-5c4adf1b-9120-4c29-9d56-2b0b4b8bb0c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a0de9ca12746be893b97d3cd11b826'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:58.929093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-5c4adf1b-9120-4c29-9d56-2b0b4b8bb0c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a0de9ca12746be893b97d3cd11b826'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:58.929969 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-5c4adf1b-9120-4c29-9d56-2b0b4b8bb0c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a0de9ca12746be893b97d3cd11b826'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:58.937805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-5c4adf1b-9120-4c29-9d56-2b0b4b8bb0c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a0de9ca12746be893b97d3cd11b826'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:58.961038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1341/2662] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:00:58 2022] DELETE /identity/v3/users/c3a0de9ca12746be893b97d3cd11b826 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:00:59.144551 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a4089-a7ba-4276-9d2c-6464b84bcd81 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:00:59.156103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a4089-a7ba-4276-9d2c-6464b84bcd81 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:00:59.228272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:00:59.229312 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:00:59.229513 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:00:59.229697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] PATH_INFO: `/v3/projects/e2695816a7fb4bd7ab0ea420b156e6e6` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:00:59.231567 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2695816a7fb4bd7ab0ea420b156e6e6)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:00:59.233522 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:00:59.282226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2695816a7fb4bd7ab0ea420b156e6e6', 'request_id': 'req-0cb3a758-1cef-4293-a6d2-ab198398711e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.288597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2695816a7fb4bd7ab0ea420b156e6e6', 'request_id': 'req-0cb3a758-1cef-4293-a6d2-ab198398711e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.296593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2695816a7fb4bd7ab0ea420b156e6e6', 'request_id': 'req-0cb3a758-1cef-4293-a6d2-ab198398711e'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.308340 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] The token cache is being invalidate because project e2695816a7fb4bd7ab0ea420b156e6e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:00:59.308613 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.309285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.311154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1322/2663] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:00:59 2022] DELETE /identity/v3/projects/e2695816a7fb4bd7ab0ea420b156e6e6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:00:59.315613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4adf1b-9120-4c29-9d56-2b0b4b8bb0c5 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:00:59.317150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-5c4adf1b-9120-4c29-9d56-2b0b4b8bb0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:00:59.331171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4adf1b-9120-4c29-9d56-2b0b4b8bb0c5 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:00:59.389128 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6dc93-7c31-4834-904c-97e451615904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:00:59.390010 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6dc93-7c31-4834-904c-97e451615904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:00:59.390295 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6dc93-7c31-4834-904c-97e451615904 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:00:59.390573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6dc93-7c31-4834-904c-97e451615904 admin admin] PATH_INFO: `/v3/users/42a643613f144989b42be5865e82e4f2` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:00:59.391842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6dc93-7c31-4834-904c-97e451615904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42a643613f144989b42be5865e82e4f2)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:00:59.393806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6dc93-7c31-4834-904c-97e451615904 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:00:59.440363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-41a6dc93-7c31-4834-904c-97e451615904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42a643613f144989b42be5865e82e4f2'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.452170 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-41a6dc93-7c31-4834-904c-97e451615904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42a643613f144989b42be5865e82e4f2'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.452900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-41a6dc93-7c31-4834-904c-97e451615904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42a643613f144989b42be5865e82e4f2'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.453639 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-41a6dc93-7c31-4834-904c-97e451615904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42a643613f144989b42be5865e82e4f2'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.460575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-41a6dc93-7c31-4834-904c-97e451615904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42a643613f144989b42be5865e82e4f2'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.474643 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1342/2664] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:00:59 2022] DELETE /identity/v3/users/42a643613f144989b42be5865e82e4f2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:00:59.665842 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:00:59.671236 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:00:59.691051 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3a758-1cef-4293-a6d2-ab198398711e admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:00:59.762524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:00:59.763662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:00:59.765316 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:00:59.765884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] PATH_INFO: `/v3/projects/1db44b16e87747aeab1ff242b1556794` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:00:59.767495 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1db44b16e87747aeab1ff242b1556794)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:00:59.769697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:00:59.829593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1db44b16e87747aeab1ff242b1556794', 'request_id': 'req-0ffa9b22-269c-440e-b5c0-9fb54985d67f'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.836255 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1db44b16e87747aeab1ff242b1556794', 'request_id': 'req-0ffa9b22-269c-440e-b5c0-9fb54985d67f'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.845541 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1db44b16e87747aeab1ff242b1556794', 'request_id': 'req-0ffa9b22-269c-440e-b5c0-9fb54985d67f'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.859254 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] The token cache is being invalidate because project 1db44b16e87747aeab1ff242b1556794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:00:59.859543 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.860193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:00:59.863907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1323/2665] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:00:59 2022] DELETE /identity/v3/projects/1db44b16e87747aeab1ff242b1556794 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:01:40.197402 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6dc93-7c31-4834-904c-97e451615904 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:01:40.198837 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-41a6dc93-7c31-4834-904c-97e451615904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:01:40.257599 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6dc93-7c31-4834-904c-97e451615904 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:01:40.303162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa47c32-0dd1-4611-9eaf-35caecd815a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:01:40.303793 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa47c32-0dd1-4611-9eaf-35caecd815a8 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:01:40.304073 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa47c32-0dd1-4611-9eaf-35caecd815a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:01:40.304345 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa47c32-0dd1-4611-9eaf-35caecd815a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:01:40.306087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-1fa47c32-0dd1-4611-9eaf-35caecd815a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:01:40.372108 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa47c32-0dd1-4611-9eaf-35caecd815a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:01:40.373534 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa47c32-0dd1-4611-9eaf-35caecd815a8 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:01:40.384141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1343/2666] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:01:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:01:41.573943 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:01:41.578463 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:01:41.592262 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa9b22-269c-440e-b5c0-9fb54985d67f admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:01:41.596126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71cf6e-7cf6-4611-b45a-059e9945c7e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:01:41.597470 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71cf6e-7cf6-4611-b45a-059e9945c7e5 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:01:41.597827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71cf6e-7cf6-4611-b45a-059e9945c7e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:01:41.598194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71cf6e-7cf6-4611-b45a-059e9945c7e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:01:41.600981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-7b71cf6e-7cf6-4611-b45a-059e9945c7e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:01:41.618215 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71cf6e-7cf6-4611-b45a-059e9945c7e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:01:41.619689 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71cf6e-7cf6-4611-b45a-059e9945c7e5 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:01:41.630940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1324/2667] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:01:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 22:01:44.640659 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104d0df-95f1-432a-9439-82ea4c950742 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:01:44.641188 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104d0df-95f1-432a-9439-82ea4c950742 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:01:44.641297 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104d0df-95f1-432a-9439-82ea4c950742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:01:44.658624 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a104d0df-95f1-432a-9439-82ea4c950742 None None] MFA Rules not processed for user `dddbe23734ee4cee866d58e3314007ea`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:01:44.660422 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a104d0df-95f1-432a-9439-82ea4c950742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:01:44.715314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1344/2668] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:01:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:06.545011 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71cf6e-7cf6-4611-b45a-059e9945c7e5 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:06.546598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-7b71cf6e-7cf6-4611-b45a-059e9945c7e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:06.612737 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71cf6e-7cf6-4611-b45a-059e9945c7e5 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:06.617205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac23c3-b431-47ce-950a-dd8abb554d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:06.618061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac23c3-b431-47ce-950a-dd8abb554d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:06.618342 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac23c3-b431-47ce-950a-dd8abb554d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:06.618615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac23c3-b431-47ce-950a-dd8abb554d4f admin admin] PATH_INFO: `/v3/users/ce6d80446f54487387c7f58eeb3c83c1` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:06.619843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac23c3-b431-47ce-950a-dd8abb554d4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce6d80446f54487387c7f58eeb3c83c1)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:06.621762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac23c3-b431-47ce-950a-dd8abb554d4f admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:06.667719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-45ac23c3-b431-47ce-950a-dd8abb554d4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce6d80446f54487387c7f58eeb3c83c1'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:06.681687 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-45ac23c3-b431-47ce-950a-dd8abb554d4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce6d80446f54487387c7f58eeb3c83c1'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:06.682785 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-45ac23c3-b431-47ce-950a-dd8abb554d4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce6d80446f54487387c7f58eeb3c83c1'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:06.683759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-45ac23c3-b431-47ce-950a-dd8abb554d4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce6d80446f54487387c7f58eeb3c83c1'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:06.692900 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-45ac23c3-b431-47ce-950a-dd8abb554d4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce6d80446f54487387c7f58eeb3c83c1'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:06.708138 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1325/2669] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:02:06 2022] DELETE /identity/v3/users/ce6d80446f54487387c7f58eeb3c83c1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:06.886833 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104d0df-95f1-432a-9439-82ea4c950742 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:06.897637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104d0df-95f1-432a-9439-82ea4c950742 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:06.959709 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743d665-224e-4941-9e3a-9ed871e7c24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:06.960560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743d665-224e-4941-9e3a-9ed871e7c24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:06.960871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743d665-224e-4941-9e3a-9ed871e7c24f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:06.961148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743d665-224e-4941-9e3a-9ed871e7c24f admin admin] PATH_INFO: `/v3/projects/a04ed773a4b44bf5b881acbdf51a8463` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:06.962323 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743d665-224e-4941-9e3a-9ed871e7c24f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a04ed773a4b44bf5b881acbdf51a8463)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:06.964282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743d665-224e-4941-9e3a-9ed871e7c24f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:07.008856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-c743d665-224e-4941-9e3a-9ed871e7c24f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a04ed773a4b44bf5b881acbdf51a8463', 'request_id': 'req-c743d665-224e-4941-9e3a-9ed871e7c24f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:07.014779 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-c743d665-224e-4941-9e3a-9ed871e7c24f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a04ed773a4b44bf5b881acbdf51a8463', 'request_id': 'req-c743d665-224e-4941-9e3a-9ed871e7c24f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:07.021458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-c743d665-224e-4941-9e3a-9ed871e7c24f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a04ed773a4b44bf5b881acbdf51a8463', 'request_id': 'req-c743d665-224e-4941-9e3a-9ed871e7c24f'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:07.031299 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-c743d665-224e-4941-9e3a-9ed871e7c24f admin admin] The token cache is being invalidate because project a04ed773a4b44bf5b881acbdf51a8463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:02:07.031629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-c743d665-224e-4941-9e3a-9ed871e7c24f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:07.032250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-c743d665-224e-4941-9e3a-9ed871e7c24f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:07.033767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1345/2670] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:02:06 2022] DELETE /identity/v3/projects/a04ed773a4b44bf5b881acbdf51a8463 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:07.040126 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7db55b-b582-4045-95da-cc51b6b37600 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.040668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7db55b-b582-4045-95da-cc51b6b37600 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.041206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7db55b-b582-4045-95da-cc51b6b37600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.063718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-4d7db55b-b582-4045-95da-cc51b6b37600 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:07.064936 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-4d7db55b-b582-4045-95da-cc51b6b37600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:07.122118 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1326/2671] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:07.130470 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acc3f1-3719-4541-a602-6d69f2c583a1 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.130725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acc3f1-3719-4541-a602-6d69f2c583a1 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.131017 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acc3f1-3719-4541-a602-6d69f2c583a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.149073 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-b9acc3f1-3719-4541-a602-6d69f2c583a1 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:07.150255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-b9acc3f1-3719-4541-a602-6d69f2c583a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:07.158134 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1346/2672] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:07.162837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7db55b-b582-4045-95da-cc51b6b37600 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:07.172359 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7db55b-b582-4045-95da-cc51b6b37600 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:07.177129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0ec12-8575-4827-9166-3c9c55116d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:07.177783 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0ec12-8575-4827-9166-3c9c55116d09 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.178071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0ec12-8575-4827-9166-3c9c55116d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.178351 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0ec12-8575-4827-9166-3c9c55116d09 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.179109 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0ec12-8575-4827-9166-3c9c55116d09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 22:02:07.179418 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0ec12-8575-4827-9166-3c9c55116d09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:07.181058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0ec12-8575-4827-9166-3c9c55116d09 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:07.192581 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1327/2673] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 22:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 22:02:07.196542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acc3f1-3719-4541-a602-6d69f2c583a1 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:07.205612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acc3f1-3719-4541-a602-6d69f2c583a1 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:07.210887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c1774-341a-482f-9a57-83a24b91804a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:07.211530 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c1774-341a-482f-9a57-83a24b91804a admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.211813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c1774-341a-482f-9a57-83a24b91804a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.212091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c1774-341a-482f-9a57-83a24b91804a admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.212770 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c1774-341a-482f-9a57-83a24b91804a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:07.214662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c1774-341a-482f-9a57-83a24b91804a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:07.239218 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1347/2674] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:02:07 2022] POST /identity/v3/projects => generated 381 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:07.243246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0ec12-8575-4827-9166-3c9c55116d09 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:07.252939 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0ec12-8575-4827-9166-3c9c55116d09 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:07.307922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c876b9f-da7c-4ef9-af11-c72e6d3d96e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:07.308586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c876b9f-da7c-4ef9-af11-c72e6d3d96e9 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.308876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c876b9f-da7c-4ef9-af11-c72e6d3d96e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.309157 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c876b9f-da7c-4ef9-af11-c72e6d3d96e9 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.309836 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c876b9f-da7c-4ef9-af11-c72e6d3d96e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:07.311800 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c876b9f-da7c-4ef9-af11-c72e6d3d96e9 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:07.330629 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1328/2675] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:02:07 2022] POST /identity/v3/users => generated 291 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:07.335181 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c1774-341a-482f-9a57-83a24b91804a admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:07.345291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c1774-341a-482f-9a57-83a24b91804a admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:07.350486 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d1f53-c7e6-4b25-aa62-3df2cd7e674e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:07.351166 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d1f53-c7e6-4b25-aa62-3df2cd7e674e admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.351453 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d1f53-c7e6-4b25-aa62-3df2cd7e674e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.351761 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d1f53-c7e6-4b25-aa62-3df2cd7e674e admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.352478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d1f53-c7e6-4b25-aa62-3df2cd7e674e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:07.354083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d1f53-c7e6-4b25-aa62-3df2cd7e674e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:07.365871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1348/2676] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:07.369800 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c876b9f-da7c-4ef9-af11-c72e6d3d96e9 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:07.379745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c876b9f-da7c-4ef9-af11-c72e6d3d96e9 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:07.384414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e330f-2259-4d25-bd3f-a8fc5bf4cbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:07.385004 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e330f-2259-4d25-bd3f-a8fc5bf4cbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.385289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e330f-2259-4d25-bd3f-a8fc5bf4cbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.385569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e330f-2259-4d25-bd3f-a8fc5bf4cbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.386291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e330f-2259-4d25-bd3f-a8fc5bf4cbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:07.387918 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e330f-2259-4d25-bd3f-a8fc5bf4cbe3 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:07.398151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1329/2677] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:07.402528 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d1f53-c7e6-4b25-aa62-3df2cd7e674e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:07.412885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d1f53-c7e6-4b25-aa62-3df2cd7e674e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:07.417510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4e847-bf27-46d4-b6ec-1f823f1cd61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:07.418374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4e847-bf27-46d4-b6ec-1f823f1cd61b admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.418657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4e847-bf27-46d4-b6ec-1f823f1cd61b admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.418968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4e847-bf27-46d4-b6ec-1f823f1cd61b admin admin] PATH_INFO: `/v3/projects/69340fb8d3f34f8caf9752e1b8c1071a/users/493a5e0c70dd4218ab11f58d25769fa2/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.433239 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4e847-bf27-46d4-b6ec-1f823f1cd61b admin admin] RBAC: Authorizing `identity:create_grant(project_id=69340fb8d3f34f8caf9752e1b8c1071a, user_id=493a5e0c70dd4218ab11f58d25769fa2, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:07.435971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4e847-bf27-46d4-b6ec-1f823f1cd61b admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:07.448559 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1349/2678] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:07 2022] PUT /identity/v3/projects/69340fb8d3f34f8caf9752e1b8c1071a/users/493a5e0c70dd4218ab11f58d25769fa2/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:07.458862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46468b5e-97ad-4208-bb3c-8ea0966c4434 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.459159 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46468b5e-97ad-4208-bb3c-8ea0966c4434 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.459442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46468b5e-97ad-4208-bb3c-8ea0966c4434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.485018 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-46468b5e-97ad-4208-bb3c-8ea0966c4434 None None] MFA Rules not processed for user `493a5e0c70dd4218ab11f58d25769fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:07.486238 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-46468b5e-97ad-4208-bb3c-8ea0966c4434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:07.546983 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1330/2679] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:07.551482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4e847-bf27-46d4-b6ec-1f823f1cd61b admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:07.610663 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4e847-bf27-46d4-b6ec-1f823f1cd61b admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:07.615449 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009ecce-de88-4268-af77-5c32bd8d24c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:07.616089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009ecce-de88-4268-af77-5c32bd8d24c8 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.616368 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009ecce-de88-4268-af77-5c32bd8d24c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.616639 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009ecce-de88-4268-af77-5c32bd8d24c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.617301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009ecce-de88-4268-af77-5c32bd8d24c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:07.619255 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009ecce-de88-4268-af77-5c32bd8d24c8 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:07.643886 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1350/2680] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:02:07 2022] POST /identity/v3/projects => generated 381 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:07.647935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46468b5e-97ad-4208-bb3c-8ea0966c4434 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:07.658425 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46468b5e-97ad-4208-bb3c-8ea0966c4434 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:07.712864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60882a20-3956-4c78-996c-021a22681788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:07.713524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60882a20-3956-4c78-996c-021a22681788 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.713810 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60882a20-3956-4c78-996c-021a22681788 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.714081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60882a20-3956-4c78-996c-021a22681788 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.714781 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60882a20-3956-4c78-996c-021a22681788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:07.716702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60882a20-3956-4c78-996c-021a22681788 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:07.735378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1331/2681] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:02:07 2022] POST /identity/v3/users => generated 297 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 22:02:07.739412 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009ecce-de88-4268-af77-5c32bd8d24c8 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:07.749324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009ecce-de88-4268-af77-5c32bd8d24c8 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:07.754306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81c686-edc4-4b54-b2f6-85e2f73167b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:07.755010 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81c686-edc4-4b54-b2f6-85e2f73167b7 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.755291 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81c686-edc4-4b54-b2f6-85e2f73167b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.755561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81c686-edc4-4b54-b2f6-85e2f73167b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.756267 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81c686-edc4-4b54-b2f6-85e2f73167b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:07.757862 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81c686-edc4-4b54-b2f6-85e2f73167b7 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:07.768030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1351/2682] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 22:02:07.772409 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60882a20-3956-4c78-996c-021a22681788 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:07.781827 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60882a20-3956-4c78-996c-021a22681788 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:07.786316 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba6a09-ec1e-4220-a0ea-631be9c5c7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:07.787206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba6a09-ec1e-4220-a0ea-631be9c5c7a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.787485 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba6a09-ec1e-4220-a0ea-631be9c5c7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.787783 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba6a09-ec1e-4220-a0ea-631be9c5c7a5 admin admin] PATH_INFO: `/v3/projects/38291da98a8d448fb52b46c7c16c713a/users/068b603b38a34182b996567b4adf7eea/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.802192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba6a09-ec1e-4220-a0ea-631be9c5c7a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38291da98a8d448fb52b46c7c16c713a, user_id=068b603b38a34182b996567b4adf7eea, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:07.804860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba6a09-ec1e-4220-a0ea-631be9c5c7a5 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:07.817656 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1332/2683] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:07 2022] PUT /identity/v3/projects/38291da98a8d448fb52b46c7c16c713a/users/068b603b38a34182b996567b4adf7eea/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:07.821531 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81c686-edc4-4b54-b2f6-85e2f73167b7 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:07.831370 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81c686-edc4-4b54-b2f6-85e2f73167b7 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:07.889701 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501f950-2793-47b9-9773-6cc7aa97f08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:07.890376 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501f950-2793-47b9-9773-6cc7aa97f08f admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.890630 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501f950-2793-47b9-9773-6cc7aa97f08f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.890969 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501f950-2793-47b9-9773-6cc7aa97f08f admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.891687 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501f950-2793-47b9-9773-6cc7aa97f08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:07.893318 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501f950-2793-47b9-9773-6cc7aa97f08f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:07.902919 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1352/2684] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:07.906947 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba6a09-ec1e-4220-a0ea-631be9c5c7a5 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:07.917045 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba6a09-ec1e-4220-a0ea-631be9c5c7a5 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:07.921542 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86109444-99fc-40bb-9b2f-59a438032335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:07.922411 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86109444-99fc-40bb-9b2f-59a438032335 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.922718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86109444-99fc-40bb-9b2f-59a438032335 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.923007 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86109444-99fc-40bb-9b2f-59a438032335 admin admin] PATH_INFO: `/v3/projects/38291da98a8d448fb52b46c7c16c713a/users/068b603b38a34182b996567b4adf7eea/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.925500 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86109444-99fc-40bb-9b2f-59a438032335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38291da98a8d448fb52b46c7c16c713a, user_id=068b603b38a34182b996567b4adf7eea, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:07.928192 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86109444-99fc-40bb-9b2f-59a438032335 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:07.937999 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1333/2685] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:07 2022] PUT /identity/v3/projects/38291da98a8d448fb52b46c7c16c713a/users/068b603b38a34182b996567b4adf7eea/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 22:02:07.949742 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03402a16-b3c2-4eea-bf41-315c6a4f5530 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:07.950029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03402a16-b3c2-4eea-bf41-315c6a4f5530 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:07.950306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03402a16-b3c2-4eea-bf41-315c6a4f5530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:07.975200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-03402a16-b3c2-4eea-bf41-315c6a4f5530 None None] MFA Rules not processed for user `068b603b38a34182b996567b4adf7eea`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:07.976373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-03402a16-b3c2-4eea-bf41-315c6a4f5530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:08.040421 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1353/2686] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:08.053087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86109444-99fc-40bb-9b2f-59a438032335 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:08.054823 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-86109444-99fc-40bb-9b2f-59a438032335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:08.108344 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86109444-99fc-40bb-9b2f-59a438032335 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:08.111908 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3bcf7-477c-4216-8c7c-d48507a1aa2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:08.113055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3bcf7-477c-4216-8c7c-d48507a1aa2a service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:08.113335 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3bcf7-477c-4216-8c7c-d48507a1aa2a service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:08.113610 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3bcf7-477c-4216-8c7c-d48507a1aa2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:08.122606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3bcf7-477c-4216-8c7c-d48507a1aa2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:08.124048 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3bcf7-477c-4216-8c7c-d48507a1aa2a service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:08.133846 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1334/2687] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:02:08.428629 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03402a16-b3c2-4eea-bf41-315c6a4f5530 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:08.437706 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03402a16-b3c2-4eea-bf41-315c6a4f5530 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:08.442819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ed36e-6eff-4d77-8480-ae18bb3722cc tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '068b603b38a34182b996567b4adf7eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38291da98a8d448fb52b46c7c16c713a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:08.443470 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ed36e-6eff-4d77-8480-ae18bb3722cc tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:08.443757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ed36e-6eff-4d77-8480-ae18bb3722cc tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:08.444037 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ed36e-6eff-4d77-8480-ae18bb3722cc tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:08.444756 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ed36e-6eff-4d77-8480-ae18bb3722cc tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 22:02:08.445062 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ed36e-6eff-4d77-8480-ae18bb3722cc tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:08.446646 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ed36e-6eff-4d77-8480-ae18bb3722cc tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:08.458814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1354/2688] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 22:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 22:02:08.462748 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3bcf7-477c-4216-8c7c-d48507a1aa2a service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:08.471677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3bcf7-477c-4216-8c7c-d48507a1aa2a service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:08.476208 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f803e-af6d-4a6f-a56d-052ef40806dd tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '068b603b38a34182b996567b4adf7eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38291da98a8d448fb52b46c7c16c713a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:08.476841 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f803e-af6d-4a6f-a56d-052ef40806dd tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:08.477125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f803e-af6d-4a6f-a56d-052ef40806dd tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:08.477402 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f803e-af6d-4a6f-a56d-052ef40806dd tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:08.478091 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f803e-af6d-4a6f-a56d-052ef40806dd tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:08.480001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f803e-af6d-4a6f-a56d-052ef40806dd tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:08.505649 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1335/2689] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:02:08 2022] POST /identity/v3/projects => generated 354 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 22:02:08.509657 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ed36e-6eff-4d77-8480-ae18bb3722cc tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:08.520197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ed36e-6eff-4d77-8480-ae18bb3722cc tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:08.574029 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308efba-eae3-4f88-bf0f-62ccd07bfac4 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '068b603b38a34182b996567b4adf7eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38291da98a8d448fb52b46c7c16c713a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:08.574678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308efba-eae3-4f88-bf0f-62ccd07bfac4 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:08.574997 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308efba-eae3-4f88-bf0f-62ccd07bfac4 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:08.575285 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308efba-eae3-4f88-bf0f-62ccd07bfac4 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:08.575985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308efba-eae3-4f88-bf0f-62ccd07bfac4 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 22:02:08.576290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308efba-eae3-4f88-bf0f-62ccd07bfac4 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:08.577885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308efba-eae3-4f88-bf0f-62ccd07bfac4 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:08.588690 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1355/2690] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 22:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 22:02:08.623430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f803e-af6d-4a6f-a56d-052ef40806dd tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:08.634462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f803e-af6d-4a6f-a56d-052ef40806dd tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:08.639189 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19b917-bf16-4a8b-9710-a6316d75a6d0 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '068b603b38a34182b996567b4adf7eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38291da98a8d448fb52b46c7c16c713a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:08.640041 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19b917-bf16-4a8b-9710-a6316d75a6d0 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:08.640359 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19b917-bf16-4a8b-9710-a6316d75a6d0 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:08.640644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19b917-bf16-4a8b-9710-a6316d75a6d0 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] PATH_INFO: `/v3/projects/1ea4c151225845d7a5af0cd3426717bd` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:08.641833 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca19b917-bf16-4a8b-9710-a6316d75a6d0 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1ea4c151225845d7a5af0cd3426717bd)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:08.643840 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca19b917-bf16-4a8b-9710-a6316d75a6d0 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:08.687325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ca19b917-bf16-4a8b-9710-a6316d75a6d0 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ea4c151225845d7a5af0cd3426717bd', 'request_id': 'req-ca19b917-bf16-4a8b-9710-a6316d75a6d0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:08.693971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ca19b917-bf16-4a8b-9710-a6316d75a6d0 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ea4c151225845d7a5af0cd3426717bd', 'request_id': 'req-ca19b917-bf16-4a8b-9710-a6316d75a6d0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:08.700136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ca19b917-bf16-4a8b-9710-a6316d75a6d0 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ea4c151225845d7a5af0cd3426717bd', 'request_id': 'req-ca19b917-bf16-4a8b-9710-a6316d75a6d0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:08.710027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ca19b917-bf16-4a8b-9710-a6316d75a6d0 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] The token cache is being invalidate because project 1ea4c151225845d7a5af0cd3426717bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:02:08.710317 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ca19b917-bf16-4a8b-9710-a6316d75a6d0 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:08.710990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ca19b917-bf16-4a8b-9710-a6316d75a6d0 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:08.712536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1336/2691] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:02:08 2022] DELETE /identity/v3/projects/1ea4c151225845d7a5af0cd3426717bd => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:08.897476 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308efba-eae3-4f88-bf0f-62ccd07bfac4 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:08.899194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-3308efba-eae3-4f88-bf0f-62ccd07bfac4 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:08.965271 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308efba-eae3-4f88-bf0f-62ccd07bfac4 tempest-VolumeQuotaClassesTest-1239786225 tempest-VolumeQuotaClassesTest-1239786225-project-admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:09.017821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2032fc-ffe7-4593-9fb7-6206a9a59b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:09.018727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2032fc-ffe7-4593-9fb7-6206a9a59b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:09.019018 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2032fc-ffe7-4593-9fb7-6206a9a59b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:09.019295 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2032fc-ffe7-4593-9fb7-6206a9a59b00 admin admin] PATH_INFO: `/v3/users/493a5e0c70dd4218ab11f58d25769fa2` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:09.020476 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2032fc-ffe7-4593-9fb7-6206a9a59b00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=493a5e0c70dd4218ab11f58d25769fa2)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:09.022405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2032fc-ffe7-4593-9fb7-6206a9a59b00 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:09.065321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1f2032fc-ffe7-4593-9fb7-6206a9a59b00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '493a5e0c70dd4218ab11f58d25769fa2'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.076536 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1f2032fc-ffe7-4593-9fb7-6206a9a59b00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '493a5e0c70dd4218ab11f58d25769fa2'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.077196 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1f2032fc-ffe7-4593-9fb7-6206a9a59b00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '493a5e0c70dd4218ab11f58d25769fa2'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.077795 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1f2032fc-ffe7-4593-9fb7-6206a9a59b00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '493a5e0c70dd4218ab11f58d25769fa2'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.088116 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1f2032fc-ffe7-4593-9fb7-6206a9a59b00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '493a5e0c70dd4218ab11f58d25769fa2'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.104571 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1356/2692] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:02:08 2022] DELETE /identity/v3/users/493a5e0c70dd4218ab11f58d25769fa2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:09.112736 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd359e-5e8c-457a-8234-72452bd6927a None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:09.113030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd359e-5e8c-457a-8234-72452bd6927a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:09.113314 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd359e-5e8c-457a-8234-72452bd6927a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:09.130424 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-d8bd359e-5e8c-457a-8234-72452bd6927a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:09.131663 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d8bd359e-5e8c-457a-8234-72452bd6927a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:09.187733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1337/2693] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:09.197592 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2032fc-ffe7-4593-9fb7-6206a9a59b00 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:09.199100 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-1f2032fc-ffe7-4593-9fb7-6206a9a59b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:09.255058 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2032fc-ffe7-4593-9fb7-6206a9a59b00 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:09.258588 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7420c-5d77-4283-bc87-4301cf18db65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:09.259244 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7420c-5d77-4283-bc87-4301cf18db65 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:09.259531 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7420c-5d77-4283-bc87-4301cf18db65 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:09.259805 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7420c-5d77-4283-bc87-4301cf18db65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:09.269047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc7420c-5d77-4283-bc87-4301cf18db65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:09.270477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc7420c-5d77-4283-bc87-4301cf18db65 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:09.279566 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1357/2694] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:02:09.543584 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd359e-5e8c-457a-8234-72452bd6927a None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:09.545131 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d8bd359e-5e8c-457a-8234-72452bd6927a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:09.557364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd359e-5e8c-457a-8234-72452bd6927a None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:09.561954 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:09.562843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:09.563132 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:09.563412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] PATH_INFO: `/v3/projects/69340fb8d3f34f8caf9752e1b8c1071a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:09.564597 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69340fb8d3f34f8caf9752e1b8c1071a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:09.567133 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:09.613080 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69340fb8d3f34f8caf9752e1b8c1071a', 'request_id': 'req-753d2c30-0be2-4064-949f-807c8a3c5c35'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.619055 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69340fb8d3f34f8caf9752e1b8c1071a', 'request_id': 'req-753d2c30-0be2-4064-949f-807c8a3c5c35'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.625976 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69340fb8d3f34f8caf9752e1b8c1071a', 'request_id': 'req-753d2c30-0be2-4064-949f-807c8a3c5c35'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.636481 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] The token cache is being invalidate because project 69340fb8d3f34f8caf9752e1b8c1071a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:02:09.636768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.637386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.639064 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1338/2695] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:02:09 2022] DELETE /identity/v3/projects/69340fb8d3f34f8caf9752e1b8c1071a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:09.642927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7420c-5d77-4283-bc87-4301cf18db65 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:09.652330 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7420c-5d77-4283-bc87-4301cf18db65 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:09.707293 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736ecea-8cd3-4bf8-b6dc-6473db95ec4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:09.708147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736ecea-8cd3-4bf8-b6dc-6473db95ec4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:09.708426 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736ecea-8cd3-4bf8-b6dc-6473db95ec4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:09.708700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736ecea-8cd3-4bf8-b6dc-6473db95ec4f admin admin] PATH_INFO: `/v3/users/068b603b38a34182b996567b4adf7eea` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:09.709969 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736ecea-8cd3-4bf8-b6dc-6473db95ec4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=068b603b38a34182b996567b4adf7eea)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:09.711954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736ecea-8cd3-4bf8-b6dc-6473db95ec4f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:09.762757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f736ecea-8cd3-4bf8-b6dc-6473db95ec4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '068b603b38a34182b996567b4adf7eea'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.774613 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f736ecea-8cd3-4bf8-b6dc-6473db95ec4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '068b603b38a34182b996567b4adf7eea'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.775315 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f736ecea-8cd3-4bf8-b6dc-6473db95ec4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '068b603b38a34182b996567b4adf7eea'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.775913 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f736ecea-8cd3-4bf8-b6dc-6473db95ec4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '068b603b38a34182b996567b4adf7eea'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.782371 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f736ecea-8cd3-4bf8-b6dc-6473db95ec4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '068b603b38a34182b996567b4adf7eea'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:09.797806 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1358/2696] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:02:09 2022] DELETE /identity/v3/users/068b603b38a34182b996567b4adf7eea => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 22:02:10.058815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:10.060261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:10.073072 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d2c30-0be2-4064-949f-807c8a3c5c35 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:10.134968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:10.135834 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.136121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.136396 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a admin admin] PATH_INFO: `/v3/projects/38291da98a8d448fb52b46c7c16c713a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:10.137620 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a admin admin] RBAC: Authorizing `identity:delete_project(project_id=38291da98a8d448fb52b46c7c16c713a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:10.139603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:10.184144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38291da98a8d448fb52b46c7c16c713a', 'request_id': 'req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:10.190420 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38291da98a8d448fb52b46c7c16c713a', 'request_id': 'req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:10.197697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38291da98a8d448fb52b46c7c16c713a', 'request_id': 'req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:10.208376 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a admin admin] The token cache is being invalidate because project 38291da98a8d448fb52b46c7c16c713a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:02:10.208666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:10.209291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d8ca4c8e-989d-4812-8c3e-77cb835ec79a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:10.210895 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1339/2697] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:02:10 2022] DELETE /identity/v3/projects/38291da98a8d448fb52b46c7c16c713a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:10.215714 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393cc63-a33d-4bc6-8140-d1cfab990a80 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.216228 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393cc63-a33d-4bc6-8140-d1cfab990a80 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.216341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393cc63-a33d-4bc6-8140-d1cfab990a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:10.233928 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-8393cc63-a33d-4bc6-8140-d1cfab990a80 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:10.235148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-8393cc63-a33d-4bc6-8140-d1cfab990a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:10.290515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1359/2698] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:10.298064 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886886c-c6d8-4d91-a850-8c2fdaf15962 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.298352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886886c-c6d8-4d91-a850-8c2fdaf15962 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.298631 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886886c-c6d8-4d91-a850-8c2fdaf15962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:10.316598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-e886886c-c6d8-4d91-a850-8c2fdaf15962 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:10.317739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-e886886c-c6d8-4d91-a850-8c2fdaf15962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:10.325565 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1340/2699] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:10.329718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393cc63-a33d-4bc6-8140-d1cfab990a80 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:10.339634 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393cc63-a33d-4bc6-8140-d1cfab990a80 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:10.344221 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f20e5-caec-41ca-8575-733eb07bb4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:10.344867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f20e5-caec-41ca-8575-733eb07bb4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.345152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f20e5-caec-41ca-8575-733eb07bb4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.345427 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f20e5-caec-41ca-8575-733eb07bb4c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:10.346141 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f20e5-caec-41ca-8575-733eb07bb4c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 22:02:10.347087 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f20e5-caec-41ca-8575-733eb07bb4c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:10.348695 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f20e5-caec-41ca-8575-733eb07bb4c1 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:10.360896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1360/2700] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 22:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 22:02:10.365480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886886c-c6d8-4d91-a850-8c2fdaf15962 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:10.374718 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886886c-c6d8-4d91-a850-8c2fdaf15962 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:10.379305 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa60aa-5fdd-4cf2-8ae4-8bde81df8340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:10.379944 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa60aa-5fdd-4cf2-8ae4-8bde81df8340 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.380228 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa60aa-5fdd-4cf2-8ae4-8bde81df8340 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.380508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa60aa-5fdd-4cf2-8ae4-8bde81df8340 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:10.381200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa60aa-5fdd-4cf2-8ae4-8bde81df8340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:10.383119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa60aa-5fdd-4cf2-8ae4-8bde81df8340 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:10.407387 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1341/2701] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:02:10 2022] POST /identity/v3/projects => generated 387 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:10.412129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f20e5-caec-41ca-8575-733eb07bb4c1 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:10.421971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f20e5-caec-41ca-8575-733eb07bb4c1 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:10.476461 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51d8df-0b42-4acd-bd3f-105c8682ef9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:10.477121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51d8df-0b42-4acd-bd3f-105c8682ef9d admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.477408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51d8df-0b42-4acd-bd3f-105c8682ef9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.477689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51d8df-0b42-4acd-bd3f-105c8682ef9d admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:10.478378 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51d8df-0b42-4acd-bd3f-105c8682ef9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:10.480636 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51d8df-0b42-4acd-bd3f-105c8682ef9d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:10.499638 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1361/2702] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:02:10 2022] POST /identity/v3/users => generated 294 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:10.504384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa60aa-5fdd-4cf2-8ae4-8bde81df8340 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:10.514367 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa60aa-5fdd-4cf2-8ae4-8bde81df8340 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:10.519505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8f512-5c28-428e-9d80-ee524f450208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:10.520153 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8f512-5c28-428e-9d80-ee524f450208 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.520438 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8f512-5c28-428e-9d80-ee524f450208 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.520719 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8f512-5c28-428e-9d80-ee524f450208 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:10.521439 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8f512-5c28-428e-9d80-ee524f450208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:10.523082 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8f512-5c28-428e-9d80-ee524f450208 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:10.533110 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1342/2703] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:10.536842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51d8df-0b42-4acd-bd3f-105c8682ef9d admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:10.546493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51d8df-0b42-4acd-bd3f-105c8682ef9d admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:10.551316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fde277-3f61-42a2-9a19-a932cc86546f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:10.551959 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fde277-3f61-42a2-9a19-a932cc86546f admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.552248 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fde277-3f61-42a2-9a19-a932cc86546f admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.552526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fde277-3f61-42a2-9a19-a932cc86546f admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:10.553236 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fde277-3f61-42a2-9a19-a932cc86546f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:10.554888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fde277-3f61-42a2-9a19-a932cc86546f admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:10.564562 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1362/2704] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:10.568987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8f512-5c28-428e-9d80-ee524f450208 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:10.578370 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8f512-5c28-428e-9d80-ee524f450208 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:10.583561 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83504e-5254-470a-ab9f-5b5011aee7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:10.584419 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83504e-5254-470a-ab9f-5b5011aee7c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.584705 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83504e-5254-470a-ab9f-5b5011aee7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.584983 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83504e-5254-470a-ab9f-5b5011aee7c0 admin admin] PATH_INFO: `/v3/projects/c8ff533b6a0144b091c724a66fc3f80b/users/a33b3e4a66a449ea85795c073bfae88f/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:10.598931 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83504e-5254-470a-ab9f-5b5011aee7c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ff533b6a0144b091c724a66fc3f80b, user_id=a33b3e4a66a449ea85795c073bfae88f, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:10.601593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83504e-5254-470a-ab9f-5b5011aee7c0 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:10.613964 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1343/2705] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:10 2022] PUT /identity/v3/projects/c8ff533b6a0144b091c724a66fc3f80b/users/a33b3e4a66a449ea85795c073bfae88f/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:10.624038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e9ec6-3e5d-4a8e-a97a-7bb67d0182ce None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.624361 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e9ec6-3e5d-4a8e-a97a-7bb67d0182ce None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.624646 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e9ec6-3e5d-4a8e-a97a-7bb67d0182ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:10.651090 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-661e9ec6-3e5d-4a8e-a97a-7bb67d0182ce None None] MFA Rules not processed for user `a33b3e4a66a449ea85795c073bfae88f`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:10.652316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-661e9ec6-3e5d-4a8e-a97a-7bb67d0182ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:10.715051 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1363/2706] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 22:02:10.719482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83504e-5254-470a-ab9f-5b5011aee7c0 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:10.779364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83504e-5254-470a-ab9f-5b5011aee7c0 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:10.784025 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fa022-2dc5-41ec-8dbb-8f3c29f0b04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:10.784678 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fa022-2dc5-41ec-8dbb-8f3c29f0b04e admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.784967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fa022-2dc5-41ec-8dbb-8f3c29f0b04e admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.785254 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fa022-2dc5-41ec-8dbb-8f3c29f0b04e admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:10.785937 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fa022-2dc5-41ec-8dbb-8f3c29f0b04e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:10.787880 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fa022-2dc5-41ec-8dbb-8f3c29f0b04e admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:10.812125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1344/2707] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:02:10 2022] POST /identity/v3/projects => generated 387 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:10.816750 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e9ec6-3e5d-4a8e-a97a-7bb67d0182ce None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:10.827348 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e9ec6-3e5d-4a8e-a97a-7bb67d0182ce None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:10.880718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632d561-622a-4949-9d13-9610d0da8adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:10.881371 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632d561-622a-4949-9d13-9610d0da8adc admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.881655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632d561-622a-4949-9d13-9610d0da8adc admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.881933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632d561-622a-4949-9d13-9610d0da8adc admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:10.882612 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632d561-622a-4949-9d13-9610d0da8adc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:10.884566 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632d561-622a-4949-9d13-9610d0da8adc admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:10.904646 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1364/2708] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:02:10 2022] POST /identity/v3/users => generated 294 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 22:02:10.908532 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fa022-2dc5-41ec-8dbb-8f3c29f0b04e admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:10.918645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fa022-2dc5-41ec-8dbb-8f3c29f0b04e admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:10.923560 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3fa7d-807a-4291-b705-23e0cc74a19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:10.924202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3fa7d-807a-4291-b705-23e0cc74a19d admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.925027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3fa7d-807a-4291-b705-23e0cc74a19d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.925308 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3fa7d-807a-4291-b705-23e0cc74a19d admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:10.926036 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3fa7d-807a-4291-b705-23e0cc74a19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:10.927677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3fa7d-807a-4291-b705-23e0cc74a19d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:10.938134 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1345/2709] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:10.942418 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632d561-622a-4949-9d13-9610d0da8adc admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:10.952156 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632d561-622a-4949-9d13-9610d0da8adc admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:10.956734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c8b81-b04c-44b9-bb9d-18af51342573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:10.957372 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c8b81-b04c-44b9-bb9d-18af51342573 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.957655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c8b81-b04c-44b9-bb9d-18af51342573 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.957933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c8b81-b04c-44b9-bb9d-18af51342573 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:10.958672 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c8b81-b04c-44b9-bb9d-18af51342573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:10.960384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c8b81-b04c-44b9-bb9d-18af51342573 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:10.970000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1365/2710] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:10.973678 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3fa7d-807a-4291-b705-23e0cc74a19d admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:10.984408 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3fa7d-807a-4291-b705-23e0cc74a19d admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:10.991269 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff50ef0-66ba-4a83-a946-942b4a24ebe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:10.992128 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff50ef0-66ba-4a83-a946-942b4a24ebe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:10.992412 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff50ef0-66ba-4a83-a946-942b4a24ebe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:10.992690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff50ef0-66ba-4a83-a946-942b4a24ebe1 admin admin] PATH_INFO: `/v3/projects/2ccf72beb8a74be0aca7c42f39a4e261/users/49636a0e71fc4b008fbf66b422a032af/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:11.007564 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff50ef0-66ba-4a83-a946-942b4a24ebe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ccf72beb8a74be0aca7c42f39a4e261, user_id=49636a0e71fc4b008fbf66b422a032af, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:11.010603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff50ef0-66ba-4a83-a946-942b4a24ebe1 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:11.023483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1346/2711] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:10 2022] PUT /identity/v3/projects/2ccf72beb8a74be0aca7c42f39a4e261/users/49636a0e71fc4b008fbf66b422a032af/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:11.034588 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a1cb6-1c9b-4d9b-b049-78b051a6f5f9 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:11.034933 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a1cb6-1c9b-4d9b-b049-78b051a6f5f9 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:11.035215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a1cb6-1c9b-4d9b-b049-78b051a6f5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:11.061151 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a72a1cb6-1c9b-4d9b-b049-78b051a6f5f9 None None] MFA Rules not processed for user `49636a0e71fc4b008fbf66b422a032af`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:11.062360 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a72a1cb6-1c9b-4d9b-b049-78b051a6f5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:11.123633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1366/2712] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:11.127943 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff50ef0-66ba-4a83-a946-942b4a24ebe1 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:11.196044 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff50ef0-66ba-4a83-a946-942b4a24ebe1 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:11.200673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6d3fb-293a-4fc3-b5d7-052ce4934081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:11.201378 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6d3fb-293a-4fc3-b5d7-052ce4934081 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:11.201663 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6d3fb-293a-4fc3-b5d7-052ce4934081 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:11.201941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6d3fb-293a-4fc3-b5d7-052ce4934081 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:11.202650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6d3fb-293a-4fc3-b5d7-052ce4934081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:11.204609 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6d3fb-293a-4fc3-b5d7-052ce4934081 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:11.228672 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1347/2713] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:02:11 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:11.232926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a1cb6-1c9b-4d9b-b049-78b051a6f5f9 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:11.244021 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a1cb6-1c9b-4d9b-b049-78b051a6f5f9 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:11.296704 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1daf0e-655e-44ab-8399-e9dedec9c3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:11.297389 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1daf0e-655e-44ab-8399-e9dedec9c3bd admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:11.297772 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1daf0e-655e-44ab-8399-e9dedec9c3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:11.298159 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1daf0e-655e-44ab-8399-e9dedec9c3bd admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:11.298889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1daf0e-655e-44ab-8399-e9dedec9c3bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:11.301450 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1daf0e-655e-44ab-8399-e9dedec9c3bd admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:11.320485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1367/2714] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:02:11 2022] POST /identity/v3/users => generated 300 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:11.324473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6d3fb-293a-4fc3-b5d7-052ce4934081 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:11.334441 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6d3fb-293a-4fc3-b5d7-052ce4934081 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:11.339487 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb3f03-72ae-4708-ba57-70a3d89a7c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:11.340140 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3f03-72ae-4708-ba57-70a3d89a7c82 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:11.340429 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3f03-72ae-4708-ba57-70a3d89a7c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:11.340779 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3f03-72ae-4708-ba57-70a3d89a7c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:11.341534 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb3f03-72ae-4708-ba57-70a3d89a7c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:11.343184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb3f03-72ae-4708-ba57-70a3d89a7c82 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:11.353914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1348/2715] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:11.358183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1daf0e-655e-44ab-8399-e9dedec9c3bd admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:11.368465 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1daf0e-655e-44ab-8399-e9dedec9c3bd admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:11.372979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733492db-f614-4ad0-ab46-8d29c3fb0a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:11.373896 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733492db-f614-4ad0-ab46-8d29c3fb0a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:11.374127 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733492db-f614-4ad0-ab46-8d29c3fb0a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:11.374399 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733492db-f614-4ad0-ab46-8d29c3fb0a9d admin admin] PATH_INFO: `/v3/projects/b32028353ddb4703a7d228463b191e56/users/43963c9b7d3d40a2b43cea33a2f1a660/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:11.388714 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733492db-f614-4ad0-ab46-8d29c3fb0a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32028353ddb4703a7d228463b191e56, user_id=43963c9b7d3d40a2b43cea33a2f1a660, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:11.391416 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733492db-f614-4ad0-ab46-8d29c3fb0a9d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:11.404024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1368/2716] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:11 2022] PUT /identity/v3/projects/b32028353ddb4703a7d228463b191e56/users/43963c9b7d3d40a2b43cea33a2f1a660/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:11.408073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb3f03-72ae-4708-ba57-70a3d89a7c82 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:11.420645 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb3f03-72ae-4708-ba57-70a3d89a7c82 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:11.477205 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870807c-1417-4c8c-96e4-33ee84d36111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:11.477863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870807c-1417-4c8c-96e4-33ee84d36111 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:11.478144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870807c-1417-4c8c-96e4-33ee84d36111 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:11.478419 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870807c-1417-4c8c-96e4-33ee84d36111 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:11.479171 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870807c-1417-4c8c-96e4-33ee84d36111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:11.481023 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870807c-1417-4c8c-96e4-33ee84d36111 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:11.491003 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1349/2717] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:11.495111 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733492db-f614-4ad0-ab46-8d29c3fb0a9d admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:11.505319 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733492db-f614-4ad0-ab46-8d29c3fb0a9d admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:11.509889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd5870-56c3-4363-893f-b64bc5779751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:11.510784 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd5870-56c3-4363-893f-b64bc5779751 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:11.511084 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd5870-56c3-4363-893f-b64bc5779751 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:11.511357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd5870-56c3-4363-893f-b64bc5779751 admin admin] PATH_INFO: `/v3/projects/b32028353ddb4703a7d228463b191e56/users/43963c9b7d3d40a2b43cea33a2f1a660/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:11.513320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd5870-56c3-4363-893f-b64bc5779751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32028353ddb4703a7d228463b191e56, user_id=43963c9b7d3d40a2b43cea33a2f1a660, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:11.515969 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd5870-56c3-4363-893f-b64bc5779751 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:11.525754 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1369/2718] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:11 2022] PUT /identity/v3/projects/b32028353ddb4703a7d228463b191e56/users/43963c9b7d3d40a2b43cea33a2f1a660/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 22:02:11.535885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5b72c-369f-4406-a24b-4b79e0741b64 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:11.536173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5b72c-369f-4406-a24b-4b79e0741b64 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:11.536453 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5b72c-369f-4406-a24b-4b79e0741b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:11.562010 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-bff5b72c-369f-4406-a24b-4b79e0741b64 None None] MFA Rules not processed for user `43963c9b7d3d40a2b43cea33a2f1a660`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:11.563225 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-bff5b72c-369f-4406-a24b-4b79e0741b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:11.627017 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1350/2719] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:11.638171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd5870-56c3-4363-893f-b64bc5779751 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:11.639881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-2fbd5870-56c3-4363-893f-b64bc5779751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:11.695700 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd5870-56c3-4363-893f-b64bc5779751 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:11.699342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18377306-f343-4217-a931-963b3286744f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:11.699971 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377306-f343-4217-a931-963b3286744f service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:11.700249 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377306-f343-4217-a931-963b3286744f service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:11.700518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377306-f343-4217-a931-963b3286744f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:11.709649 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18377306-f343-4217-a931-963b3286744f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:11.711094 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18377306-f343-4217-a931-963b3286744f service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:11.720078 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1370/2720] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:02:12.187114 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5b72c-369f-4406-a24b-4b79e0741b64 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:12.188538 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-bff5b72c-369f-4406-a24b-4b79e0741b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:12.200844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5b72c-369f-4406-a24b-4b79e0741b64 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:12.204483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12082f3d-c309-47fc-a2b2-87d3d0cc67d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:12.205121 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12082f3d-c309-47fc-a2b2-87d3d0cc67d0 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:12.205409 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12082f3d-c309-47fc-a2b2-87d3d0cc67d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:12.205693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12082f3d-c309-47fc-a2b2-87d3d0cc67d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:12.261030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12082f3d-c309-47fc-a2b2-87d3d0cc67d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:12.262452 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12082f3d-c309-47fc-a2b2-87d3d0cc67d0 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:12.271586 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1351/2721] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:02:16.199658 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18377306-f343-4217-a931-963b3286744f service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:16.208803 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18377306-f343-4217-a931-963b3286744f service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:16.212502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c2914-0533-44d0-be85-56cbfe688278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:16.213124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c2914-0533-44d0-be85-56cbfe688278 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:16.213404 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c2914-0533-44d0-be85-56cbfe688278 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:16.213679 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c2914-0533-44d0-be85-56cbfe688278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:16.271156 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c2914-0533-44d0-be85-56cbfe688278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:16.272521 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c2914-0533-44d0-be85-56cbfe688278 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:16.281201 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1371/2722] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:02:18.299245 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12082f3d-c309-47fc-a2b2-87d3d0cc67d0 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:18.360917 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12082f3d-c309-47fc-a2b2-87d3d0cc67d0 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:18.367525 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da5f22-7200-4292-8633-3572289e1573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:18.368425 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da5f22-7200-4292-8633-3572289e1573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:18.368822 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da5f22-7200-4292-8633-3572289e1573 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:18.369106 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da5f22-7200-4292-8633-3572289e1573 admin admin] PATH_INFO: `/v3/users/a33b3e4a66a449ea85795c073bfae88f` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:18.370356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da5f22-7200-4292-8633-3572289e1573 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a33b3e4a66a449ea85795c073bfae88f)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:18.372358 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da5f22-7200-4292-8633-3572289e1573 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:18.417569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-14da5f22-7200-4292-8633-3572289e1573 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a33b3e4a66a449ea85795c073bfae88f'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:18.429027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-14da5f22-7200-4292-8633-3572289e1573 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a33b3e4a66a449ea85795c073bfae88f'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:18.429714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-14da5f22-7200-4292-8633-3572289e1573 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a33b3e4a66a449ea85795c073bfae88f'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:18.430336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-14da5f22-7200-4292-8633-3572289e1573 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a33b3e4a66a449ea85795c073bfae88f'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:18.436978 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-14da5f22-7200-4292-8633-3572289e1573 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a33b3e4a66a449ea85795c073bfae88f'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:18.449734 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1352/2723] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:02:18 2022] DELETE /identity/v3/users/a33b3e4a66a449ea85795c073bfae88f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:18.454917 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebd907-84e5-42c8-bc17-d0303dec71cb None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:18.455483 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebd907-84e5-42c8-bc17-d0303dec71cb None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:18.455619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebd907-84e5-42c8-bc17-d0303dec71cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:18.473709 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a9ebd907-84e5-42c8-bc17-d0303dec71cb None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:18.480635 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a9ebd907-84e5-42c8-bc17-d0303dec71cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:18.537413 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1372/2724] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:18.547312 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da5f22-7200-4292-8633-3572289e1573 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:18.549020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-14da5f22-7200-4292-8633-3572289e1573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:18.604062 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da5f22-7200-4292-8633-3572289e1573 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:18.608043 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925fbff-9672-45f6-b7a2-dbe2d4bff99c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:18.608756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925fbff-9672-45f6-b7a2-dbe2d4bff99c service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:18.609046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925fbff-9672-45f6-b7a2-dbe2d4bff99c service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:18.609333 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925fbff-9672-45f6-b7a2-dbe2d4bff99c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:18.618345 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925fbff-9672-45f6-b7a2-dbe2d4bff99c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:18.619845 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925fbff-9672-45f6-b7a2-dbe2d4bff99c service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:18.629194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1353/2725] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 22:02:18.902748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebd907-84e5-42c8-bc17-d0303dec71cb None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:18.904505 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a9ebd907-84e5-42c8-bc17-d0303dec71cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:18.918897 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebd907-84e5-42c8-bc17-d0303dec71cb None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:18.924464 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:18.925406 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:18.925628 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:18.925926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] PATH_INFO: `/v3/projects/c8ff533b6a0144b091c724a66fc3f80b` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:18.927211 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ff533b6a0144b091c724a66fc3f80b)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:18.929110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:18.974661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ff533b6a0144b091c724a66fc3f80b', 'request_id': 'req-6d14f527-2026-4929-8093-3e10af4ea7d2'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:18.980789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ff533b6a0144b091c724a66fc3f80b', 'request_id': 'req-6d14f527-2026-4929-8093-3e10af4ea7d2'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:18.987716 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ff533b6a0144b091c724a66fc3f80b', 'request_id': 'req-6d14f527-2026-4929-8093-3e10af4ea7d2'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:18.997511 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] The token cache is being invalidate because project c8ff533b6a0144b091c724a66fc3f80b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:02:18.997800 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:18.998415 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:18.999953 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1373/2726] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:02:18 2022] DELETE /identity/v3/projects/c8ff533b6a0144b091c724a66fc3f80b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:19.004572 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925fbff-9672-45f6-b7a2-dbe2d4bff99c service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:19.013925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925fbff-9672-45f6-b7a2-dbe2d4bff99c service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:19.068820 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f452da-73c7-452b-be53-38b03068cc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:19.069693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f452da-73c7-452b-be53-38b03068cc19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:19.069981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f452da-73c7-452b-be53-38b03068cc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:19.070263 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f452da-73c7-452b-be53-38b03068cc19 admin admin] PATH_INFO: `/v3/users/49636a0e71fc4b008fbf66b422a032af` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:19.071473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f452da-73c7-452b-be53-38b03068cc19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49636a0e71fc4b008fbf66b422a032af)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:19.073387 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f452da-73c7-452b-be53-38b03068cc19 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:19.117774 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-32f452da-73c7-452b-be53-38b03068cc19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49636a0e71fc4b008fbf66b422a032af'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.129467 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-32f452da-73c7-452b-be53-38b03068cc19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49636a0e71fc4b008fbf66b422a032af'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.130129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-32f452da-73c7-452b-be53-38b03068cc19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49636a0e71fc4b008fbf66b422a032af'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.130756 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-32f452da-73c7-452b-be53-38b03068cc19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49636a0e71fc4b008fbf66b422a032af'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.137340 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-32f452da-73c7-452b-be53-38b03068cc19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49636a0e71fc4b008fbf66b422a032af'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.150656 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1354/2727] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:02:19 2022] DELETE /identity/v3/users/49636a0e71fc4b008fbf66b422a032af => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 22:02:19.411734 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:19.413215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:19.426884 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14f527-2026-4929-8093-3e10af4ea7d2 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:19.481780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:19.482621 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:19.482929 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:19.483204 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] PATH_INFO: `/v3/projects/2ccf72beb8a74be0aca7c42f39a4e261` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:19.484361 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ccf72beb8a74be0aca7c42f39a4e261)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:19.486247 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:19.530314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ccf72beb8a74be0aca7c42f39a4e261', 'request_id': 'req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.536819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ccf72beb8a74be0aca7c42f39a4e261', 'request_id': 'req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.543881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ccf72beb8a74be0aca7c42f39a4e261', 'request_id': 'req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.554504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] The token cache is being invalidate because project 2ccf72beb8a74be0aca7c42f39a4e261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:02:19.554825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.555442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.557003 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1374/2728] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:02:19 2022] DELETE /identity/v3/projects/2ccf72beb8a74be0aca7c42f39a4e261 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:19.560923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f452da-73c7-452b-be53-38b03068cc19 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:19.562309 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-32f452da-73c7-452b-be53-38b03068cc19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:19.574643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f452da-73c7-452b-be53-38b03068cc19 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:19.628365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c2dfa-a815-4d5b-9bae-83befaaecb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:19.629223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c2dfa-a815-4d5b-9bae-83befaaecb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:19.629507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c2dfa-a815-4d5b-9bae-83befaaecb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:19.629781 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c2dfa-a815-4d5b-9bae-83befaaecb1b admin admin] PATH_INFO: `/v3/users/43963c9b7d3d40a2b43cea33a2f1a660` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:19.630984 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c2dfa-a815-4d5b-9bae-83befaaecb1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=43963c9b7d3d40a2b43cea33a2f1a660)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:19.632887 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c2dfa-a815-4d5b-9bae-83befaaecb1b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:19.676495 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ac4c2dfa-a815-4d5b-9bae-83befaaecb1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43963c9b7d3d40a2b43cea33a2f1a660'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.688430 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ac4c2dfa-a815-4d5b-9bae-83befaaecb1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43963c9b7d3d40a2b43cea33a2f1a660'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.689089 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ac4c2dfa-a815-4d5b-9bae-83befaaecb1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43963c9b7d3d40a2b43cea33a2f1a660'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.689703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ac4c2dfa-a815-4d5b-9bae-83befaaecb1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43963c9b7d3d40a2b43cea33a2f1a660'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.696223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-ac4c2dfa-a815-4d5b-9bae-83befaaecb1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43963c9b7d3d40a2b43cea33a2f1a660'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:19.709232 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1355/2729] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:02:19 2022] DELETE /identity/v3/users/43963c9b7d3d40a2b43cea33a2f1a660 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:19.975157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:19.976580 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:19.989396 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fc18d-9e62-41ac-80bb-d88475eaf5ac admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:20.043941 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:20.044785 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.045065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.045338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe admin admin] PATH_INFO: `/v3/projects/b32028353ddb4703a7d228463b191e56` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.046487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b32028353ddb4703a7d228463b191e56)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:20.048440 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:20.090713 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b32028353ddb4703a7d228463b191e56', 'request_id': 'req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:20.096496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b32028353ddb4703a7d228463b191e56', 'request_id': 'req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:20.103816 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b32028353ddb4703a7d228463b191e56', 'request_id': 'req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:20.113860 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe admin admin] The token cache is being invalidate because project b32028353ddb4703a7d228463b191e56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:02:20.114147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:20.114781 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-03d8fea7-5cf4-48bd-8841-cfd923eb89fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:20.116280 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1375/2730] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:02:19 2022] DELETE /identity/v3/projects/b32028353ddb4703a7d228463b191e56 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 22:02:20.122650 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f8818-2b8a-4e03-8412-4d39341c0952 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.122968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f8818-2b8a-4e03-8412-4d39341c0952 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.123246 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f8818-2b8a-4e03-8412-4d39341c0952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.140838 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-346f8818-2b8a-4e03-8412-4d39341c0952 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:20.142034 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-346f8818-2b8a-4e03-8412-4d39341c0952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:20.198847 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1356/2731] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:20.207401 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a6bae-60cd-470a-8259-8c3b1a8f5ec7 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.207692 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a6bae-60cd-470a-8259-8c3b1a8f5ec7 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.207970 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a6bae-60cd-470a-8259-8c3b1a8f5ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.226162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-092a6bae-60cd-470a-8259-8c3b1a8f5ec7 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:20.227376 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-092a6bae-60cd-470a-8259-8c3b1a8f5ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:20.235601 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1376/2732] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:20.239835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f8818-2b8a-4e03-8412-4d39341c0952 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:20.249297 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f8818-2b8a-4e03-8412-4d39341c0952 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:20.254456 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa4f86-0658-41a8-b727-dfd72cdea8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:20.255141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa4f86-0658-41a8-b727-dfd72cdea8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.255427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa4f86-0658-41a8-b727-dfd72cdea8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.255708 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa4f86-0658-41a8-b727-dfd72cdea8b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.256427 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa4f86-0658-41a8-b727-dfd72cdea8b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 22:02:20.256764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa4f86-0658-41a8-b727-dfd72cdea8b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:20.258366 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa4f86-0658-41a8-b727-dfd72cdea8b9 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:20.270244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1357/2733] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 22:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 22:02:20.274279 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a6bae-60cd-470a-8259-8c3b1a8f5ec7 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:20.283215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a6bae-60cd-470a-8259-8c3b1a8f5ec7 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:20.287796 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509e12a-a33f-48d1-80a8-409cff3ca342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:20.288433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509e12a-a33f-48d1-80a8-409cff3ca342 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.288718 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509e12a-a33f-48d1-80a8-409cff3ca342 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.288994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509e12a-a33f-48d1-80a8-409cff3ca342 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.289678 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509e12a-a33f-48d1-80a8-409cff3ca342 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:20.291617 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509e12a-a33f-48d1-80a8-409cff3ca342 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:20.315550 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1377/2734] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:02:20 2022] POST /identity/v3/projects => generated 401 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 22:02:20.320046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa4f86-0658-41a8-b727-dfd72cdea8b9 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:20.329862 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa4f86-0658-41a8-b727-dfd72cdea8b9 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:20.387708 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb6b02-d2c6-46b4-90f4-fcfa19434a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:20.388374 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6b02-d2c6-46b4-90f4-fcfa19434a32 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.388661 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6b02-d2c6-46b4-90f4-fcfa19434a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.388942 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6b02-d2c6-46b4-90f4-fcfa19434a32 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.389625 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb6b02-d2c6-46b4-90f4-fcfa19434a32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:20.391594 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb6b02-d2c6-46b4-90f4-fcfa19434a32 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:20.411398 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1358/2735] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:02:20 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:20.415328 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509e12a-a33f-48d1-80a8-409cff3ca342 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:20.425344 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509e12a-a33f-48d1-80a8-409cff3ca342 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:20.429944 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1017b8-3c4b-4220-9512-9c854934cf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:20.430588 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1017b8-3c4b-4220-9512-9c854934cf76 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.430903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1017b8-3c4b-4220-9512-9c854934cf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.431183 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1017b8-3c4b-4220-9512-9c854934cf76 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.431934 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1017b8-3c4b-4220-9512-9c854934cf76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:20.433542 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1017b8-3c4b-4220-9512-9c854934cf76 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:20.444333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1378/2736] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:20.448158 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb6b02-d2c6-46b4-90f4-fcfa19434a32 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:20.458374 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb6b02-d2c6-46b4-90f4-fcfa19434a32 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:20.462952 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bbb2b-ea72-4a10-a793-1a840c5cdffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:20.463593 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bbb2b-ea72-4a10-a793-1a840c5cdffe admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.463876 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bbb2b-ea72-4a10-a793-1a840c5cdffe admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.464159 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bbb2b-ea72-4a10-a793-1a840c5cdffe admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.464873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bbb2b-ea72-4a10-a793-1a840c5cdffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:20.466469 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bbb2b-ea72-4a10-a793-1a840c5cdffe admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:20.476632 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1359/2737] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:20.480623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1017b8-3c4b-4220-9512-9c854934cf76 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:20.490217 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1017b8-3c4b-4220-9512-9c854934cf76 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:20.494792 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25e271-54ca-4f1c-afce-78a31c593421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:20.495639 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25e271-54ca-4f1c-afce-78a31c593421 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.495921 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25e271-54ca-4f1c-afce-78a31c593421 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.496200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25e271-54ca-4f1c-afce-78a31c593421 admin admin] PATH_INFO: `/v3/projects/9267539ad60249f3a480c95bf19f3c9f/users/cf6b1a6d6f5147b2889eec22b36865ff/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.510435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25e271-54ca-4f1c-afce-78a31c593421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9267539ad60249f3a480c95bf19f3c9f, user_id=cf6b1a6d6f5147b2889eec22b36865ff, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:20.513096 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25e271-54ca-4f1c-afce-78a31c593421 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:20.525975 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1379/2738] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:20 2022] PUT /identity/v3/projects/9267539ad60249f3a480c95bf19f3c9f/users/cf6b1a6d6f5147b2889eec22b36865ff/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:20.537940 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bf849-ddf4-4042-956e-5185c799d3c5 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.538230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bf849-ddf4-4042-956e-5185c799d3c5 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.538518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bf849-ddf4-4042-956e-5185c799d3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.565482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-666bf849-ddf4-4042-956e-5185c799d3c5 None None] MFA Rules not processed for user `cf6b1a6d6f5147b2889eec22b36865ff`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:20.567287 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-666bf849-ddf4-4042-956e-5185c799d3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:20.628647 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1360/2739] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:20.633060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25e271-54ca-4f1c-afce-78a31c593421 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:20.691862 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25e271-54ca-4f1c-afce-78a31c593421 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:20.696485 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd85f9-9cc6-4fad-a57c-3d578d8fc1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:20.697132 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd85f9-9cc6-4fad-a57c-3d578d8fc1da admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.697420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd85f9-9cc6-4fad-a57c-3d578d8fc1da admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.697696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd85f9-9cc6-4fad-a57c-3d578d8fc1da admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.698384 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd85f9-9cc6-4fad-a57c-3d578d8fc1da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:20.700331 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd85f9-9cc6-4fad-a57c-3d578d8fc1da admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:20.725240 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1380/2740] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:02:20 2022] POST /identity/v3/projects => generated 401 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 22:02:20.729484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bf849-ddf4-4042-956e-5185c799d3c5 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:20.740185 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bf849-ddf4-4042-956e-5185c799d3c5 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:20.793507 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571e9ea-a7fd-4b37-b89f-44f05f69e520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:20.794144 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571e9ea-a7fd-4b37-b89f-44f05f69e520 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.794426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571e9ea-a7fd-4b37-b89f-44f05f69e520 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.794721 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571e9ea-a7fd-4b37-b89f-44f05f69e520 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.795442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571e9ea-a7fd-4b37-b89f-44f05f69e520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:20.797334 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571e9ea-a7fd-4b37-b89f-44f05f69e520 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:20.817123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1361/2741] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:02:20 2022] POST /identity/v3/users => generated 307 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:20.820569 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd85f9-9cc6-4fad-a57c-3d578d8fc1da admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:20.830813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd85f9-9cc6-4fad-a57c-3d578d8fc1da admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:20.835871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcb1f0-da6c-4e71-882a-3ea7de6a0595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:20.836520 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcb1f0-da6c-4e71-882a-3ea7de6a0595 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.836812 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcb1f0-da6c-4e71-882a-3ea7de6a0595 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.837091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcb1f0-da6c-4e71-882a-3ea7de6a0595 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.837807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bcb1f0-da6c-4e71-882a-3ea7de6a0595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:20.839454 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bcb1f0-da6c-4e71-882a-3ea7de6a0595 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:20.851326 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1381/2742] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 22:02:20.855580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571e9ea-a7fd-4b37-b89f-44f05f69e520 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:20.865310 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571e9ea-a7fd-4b37-b89f-44f05f69e520 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:20.870324 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7b514-528a-4667-8c6a-82a25798b2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:20.871252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7b514-528a-4667-8c6a-82a25798b2c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.871539 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7b514-528a-4667-8c6a-82a25798b2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.871820 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7b514-528a-4667-8c6a-82a25798b2c2 admin admin] PATH_INFO: `/v3/projects/8955f2f147f042e6bd8db5feb1fe3f5c/users/378fb0c0300f400fbd07707c0792e86c/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.886195 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7b514-528a-4667-8c6a-82a25798b2c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8955f2f147f042e6bd8db5feb1fe3f5c, user_id=378fb0c0300f400fbd07707c0792e86c, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:20.888877 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7b514-528a-4667-8c6a-82a25798b2c2 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:20.901789 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1362/2743] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:20 2022] PUT /identity/v3/projects/8955f2f147f042e6bd8db5feb1fe3f5c/users/378fb0c0300f400fbd07707c0792e86c/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:20.905603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcb1f0-da6c-4e71-882a-3ea7de6a0595 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:20.916312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcb1f0-da6c-4e71-882a-3ea7de6a0595 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:20.969304 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d433ffc-ffd3-49fc-bd14-404e984ad261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:20.969954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d433ffc-ffd3-49fc-bd14-404e984ad261 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:20.970232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d433ffc-ffd3-49fc-bd14-404e984ad261 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:20.970504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d433ffc-ffd3-49fc-bd14-404e984ad261 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:20.971230 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d433ffc-ffd3-49fc-bd14-404e984ad261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:20.972838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d433ffc-ffd3-49fc-bd14-404e984ad261 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:20.982427 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1382/2744] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:20.986411 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7b514-528a-4667-8c6a-82a25798b2c2 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:20.996346 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7b514-528a-4667-8c6a-82a25798b2c2 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:21.001000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319ba8e-5c83-44a0-b8df-eb9bdb1a0af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:21.001864 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319ba8e-5c83-44a0-b8df-eb9bdb1a0af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:21.002149 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319ba8e-5c83-44a0-b8df-eb9bdb1a0af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:21.002429 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319ba8e-5c83-44a0-b8df-eb9bdb1a0af1 admin admin] PATH_INFO: `/v3/projects/8955f2f147f042e6bd8db5feb1fe3f5c/users/378fb0c0300f400fbd07707c0792e86c/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:21.004901 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319ba8e-5c83-44a0-b8df-eb9bdb1a0af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8955f2f147f042e6bd8db5feb1fe3f5c, user_id=378fb0c0300f400fbd07707c0792e86c, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:21.007598 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319ba8e-5c83-44a0-b8df-eb9bdb1a0af1 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:21.017489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1363/2745] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:20 2022] PUT /identity/v3/projects/8955f2f147f042e6bd8db5feb1fe3f5c/users/378fb0c0300f400fbd07707c0792e86c/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:21.027864 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d26d8-20ef-4c6a-8ff7-7dfcade31201 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:21.028148 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d26d8-20ef-4c6a-8ff7-7dfcade31201 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:21.028421 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d26d8-20ef-4c6a-8ff7-7dfcade31201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:21.053126 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-4d1d26d8-20ef-4c6a-8ff7-7dfcade31201 None None] MFA Rules not processed for user `378fb0c0300f400fbd07707c0792e86c`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:21.054311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-4d1d26d8-20ef-4c6a-8ff7-7dfcade31201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:21.117386 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1383/2746] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:21 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:21.128015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319ba8e-5c83-44a0-b8df-eb9bdb1a0af1 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:21.130226 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2319ba8e-5c83-44a0-b8df-eb9bdb1a0af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:21.184639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319ba8e-5c83-44a0-b8df-eb9bdb1a0af1 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:21.188294 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dff41-f320-4ce5-b373-3edf83353330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:21.188929 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dff41-f320-4ce5-b373-3edf83353330 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:21.189216 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dff41-f320-4ce5-b373-3edf83353330 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:21.189493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dff41-f320-4ce5-b373-3edf83353330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:21.198888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059dff41-f320-4ce5-b373-3edf83353330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:21.200316 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059dff41-f320-4ce5-b373-3edf83353330 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:21.210006 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1364/2747] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:21 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:02:21.282366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d26d8-20ef-4c6a-8ff7-7dfcade31201 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:21.293717 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d26d8-20ef-4c6a-8ff7-7dfcade31201 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:21.297332 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabbf1a-6e18-4d88-b47c-48f4861a2e6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:21.297969 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabbf1a-6e18-4d88-b47c-48f4861a2e6b service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:21.298526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabbf1a-6e18-4d88-b47c-48f4861a2e6b service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:21.298849 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabbf1a-6e18-4d88-b47c-48f4861a2e6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:21.356721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabbf1a-6e18-4d88-b47c-48f4861a2e6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:21.358165 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabbf1a-6e18-4d88-b47c-48f4861a2e6b service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:21.368070 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1384/2748] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:21 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:02:25.333267 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dff41-f320-4ce5-b373-3edf83353330 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:25.391633 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dff41-f320-4ce5-b373-3edf83353330 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:25.396100 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf342966-10b9-419e-be78-2e1e99c0cb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:25.396943 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf342966-10b9-419e-be78-2e1e99c0cb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:25.397224 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf342966-10b9-419e-be78-2e1e99c0cb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:25.397497 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf342966-10b9-419e-be78-2e1e99c0cb70 admin admin] PATH_INFO: `/v3/users/cf6b1a6d6f5147b2889eec22b36865ff` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:25.398658 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf342966-10b9-419e-be78-2e1e99c0cb70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf6b1a6d6f5147b2889eec22b36865ff)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:25.400585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf342966-10b9-419e-be78-2e1e99c0cb70 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:25.444523 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bf342966-10b9-419e-be78-2e1e99c0cb70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf6b1a6d6f5147b2889eec22b36865ff'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:25.456494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bf342966-10b9-419e-be78-2e1e99c0cb70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf6b1a6d6f5147b2889eec22b36865ff'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:25.457184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bf342966-10b9-419e-be78-2e1e99c0cb70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf6b1a6d6f5147b2889eec22b36865ff'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:25.457782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bf342966-10b9-419e-be78-2e1e99c0cb70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf6b1a6d6f5147b2889eec22b36865ff'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:25.464993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-bf342966-10b9-419e-be78-2e1e99c0cb70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf6b1a6d6f5147b2889eec22b36865ff'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:25.483911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1365/2749] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:02:25 2022] DELETE /identity/v3/users/cf6b1a6d6f5147b2889eec22b36865ff => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:25.491022 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd66d0-394d-4ccb-ac0d-b64314951183 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:25.491586 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd66d0-394d-4ccb-ac0d-b64314951183 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:25.491586 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd66d0-394d-4ccb-ac0d-b64314951183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:25.509057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-f1bd66d0-394d-4ccb-ac0d-b64314951183 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:25.510274 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f1bd66d0-394d-4ccb-ac0d-b64314951183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:25.566626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1385/2750] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:25.576236 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf342966-10b9-419e-be78-2e1e99c0cb70 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:25.577635 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-bf342966-10b9-419e-be78-2e1e99c0cb70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:25.634873 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf342966-10b9-419e-be78-2e1e99c0cb70 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:25.638524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167aa21-0601-4bc6-8bbf-e6c9baa898ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:25.639193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167aa21-0601-4bc6-8bbf-e6c9baa898ff service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:25.639480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167aa21-0601-4bc6-8bbf-e6c9baa898ff service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:25.639758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167aa21-0601-4bc6-8bbf-e6c9baa898ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:25.648624 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167aa21-0601-4bc6-8bbf-e6c9baa898ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:25.650052 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167aa21-0601-4bc6-8bbf-e6c9baa898ff service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:25.659411 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1366/2751] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:02:25.912673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd66d0-394d-4ccb-ac0d-b64314951183 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:25.914365 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-f1bd66d0-394d-4ccb-ac0d-b64314951183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:25.928480 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd66d0-394d-4ccb-ac0d-b64314951183 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:25.934819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:25.935887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:25.936374 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:25.936714 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] PATH_INFO: `/v3/projects/9267539ad60249f3a480c95bf19f3c9f` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:25.938122 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9267539ad60249f3a480c95bf19f3c9f)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:25.940301 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:25.984960 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9267539ad60249f3a480c95bf19f3c9f', 'request_id': 'req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:25.990902 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9267539ad60249f3a480c95bf19f3c9f', 'request_id': 'req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:25.997504 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9267539ad60249f3a480c95bf19f3c9f', 'request_id': 'req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:26.007150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] The token cache is being invalidate because project 9267539ad60249f3a480c95bf19f3c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:02:26.007376 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:26.007979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:26.009467 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1386/2752] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:02:25 2022] DELETE /identity/v3/projects/9267539ad60249f3a480c95bf19f3c9f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:26.013363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167aa21-0601-4bc6-8bbf-e6c9baa898ff service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:26.023383 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167aa21-0601-4bc6-8bbf-e6c9baa898ff service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:26.075899 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae3832-93a2-4b6c-8043-eb8057818d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:26.076754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae3832-93a2-4b6c-8043-eb8057818d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:26.077038 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae3832-93a2-4b6c-8043-eb8057818d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:26.077315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae3832-93a2-4b6c-8043-eb8057818d98 admin admin] PATH_INFO: `/v3/users/378fb0c0300f400fbd07707c0792e86c` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:26.078479 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae3832-93a2-4b6c-8043-eb8057818d98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=378fb0c0300f400fbd07707c0792e86c)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:26.080413 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae3832-93a2-4b6c-8043-eb8057818d98 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:26.124496 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5fae3832-93a2-4b6c-8043-eb8057818d98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '378fb0c0300f400fbd07707c0792e86c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:26.135493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5fae3832-93a2-4b6c-8043-eb8057818d98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '378fb0c0300f400fbd07707c0792e86c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:26.136180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5fae3832-93a2-4b6c-8043-eb8057818d98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '378fb0c0300f400fbd07707c0792e86c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:26.136776 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5fae3832-93a2-4b6c-8043-eb8057818d98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '378fb0c0300f400fbd07707c0792e86c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:26.143201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-5fae3832-93a2-4b6c-8043-eb8057818d98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '378fb0c0300f400fbd07707c0792e86c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:26.156702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1367/2753] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:02:26 2022] DELETE /identity/v3/users/378fb0c0300f400fbd07707c0792e86c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:26.420452 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:26.421986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:26.435180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb7f98-e55f-4abb-809e-2dde0d0022d5 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:26.490900 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:26.491753 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:26.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:26.492314 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a admin admin] PATH_INFO: `/v3/projects/8955f2f147f042e6bd8db5feb1fe3f5c` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:26.493441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8955f2f147f042e6bd8db5feb1fe3f5c)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:26.495387 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:26.542385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8955f2f147f042e6bd8db5feb1fe3f5c', 'request_id': 'req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:26.548117 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8955f2f147f042e6bd8db5feb1fe3f5c', 'request_id': 'req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:26.555378 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8955f2f147f042e6bd8db5feb1fe3f5c', 'request_id': 'req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:26.564887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a admin admin] The token cache is being invalidate because project 8955f2f147f042e6bd8db5feb1fe3f5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:02:26.565169 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:26.565783 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-45b6ed54-8f8e-4a7b-a7d5-8f308719061a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:26.567934 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1387/2754] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:02:26 2022] DELETE /identity/v3/projects/8955f2f147f042e6bd8db5feb1fe3f5c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:26.572973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37688d2-2f83-461b-8ccd-909c443ecdca None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:26.573428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37688d2-2f83-461b-8ccd-909c443ecdca None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:26.573523 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37688d2-2f83-461b-8ccd-909c443ecdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:26.590626 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-c37688d2-2f83-461b-8ccd-909c443ecdca None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:26.591867 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c37688d2-2f83-461b-8ccd-909c443ecdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:26.649489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1368/2755] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:26.658262 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c638f5-f6c1-499a-930c-b48f12ec177f None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:26.658561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c638f5-f6c1-499a-930c-b48f12ec177f None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:26.658877 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c638f5-f6c1-499a-930c-b48f12ec177f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:26.677194 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-a7c638f5-f6c1-499a-930c-b48f12ec177f None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:26.678394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-a7c638f5-f6c1-499a-930c-b48f12ec177f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:26.686611 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1388/2756] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:26.690799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37688d2-2f83-461b-8ccd-909c443ecdca None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:26.700982 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37688d2-2f83-461b-8ccd-909c443ecdca None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:26.705570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ba558-4371-4bb7-86b1-8491d65a818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:26.706216 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ba558-4371-4bb7-86b1-8491d65a818d admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:26.706503 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ba558-4371-4bb7-86b1-8491d65a818d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:26.706809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ba558-4371-4bb7-86b1-8491d65a818d admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:26.707599 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ba558-4371-4bb7-86b1-8491d65a818d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 22:02:26.707910 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ba558-4371-4bb7-86b1-8491d65a818d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:26.709533 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ba558-4371-4bb7-86b1-8491d65a818d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:26.720697 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1369/2757] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 22:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 22:02:26.724834 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c638f5-f6c1-499a-930c-b48f12ec177f None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:26.734208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c638f5-f6c1-499a-930c-b48f12ec177f None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:26.739419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf46ee-e984-47fd-ae78-f45be33c950a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:26.740057 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf46ee-e984-47fd-ae78-f45be33c950a admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:26.740342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf46ee-e984-47fd-ae78-f45be33c950a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:26.740619 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf46ee-e984-47fd-ae78-f45be33c950a admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:26.741305 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf46ee-e984-47fd-ae78-f45be33c950a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:26.743246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf46ee-e984-47fd-ae78-f45be33c950a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:26.767487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1389/2758] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:02:26 2022] POST /identity/v3/projects => generated 409 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:26.830599 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ba558-4371-4bb7-86b1-8491d65a818d admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:26.842315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ba558-4371-4bb7-86b1-8491d65a818d admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:26.895336 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254336ed-832e-427d-9785-f9846e0dc3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:26.895985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254336ed-832e-427d-9785-f9846e0dc3ed admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:26.896266 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254336ed-832e-427d-9785-f9846e0dc3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:26.896536 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254336ed-832e-427d-9785-f9846e0dc3ed admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:26.897203 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254336ed-832e-427d-9785-f9846e0dc3ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:26.899150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254336ed-832e-427d-9785-f9846e0dc3ed admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:26.918295 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1370/2759] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:02:26 2022] POST /identity/v3/users => generated 305 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:26.922369 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf46ee-e984-47fd-ae78-f45be33c950a admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:26.933161 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf46ee-e984-47fd-ae78-f45be33c950a admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:26.937867 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc75c8-581d-4079-abbf-a54a2882cc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:26.938509 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc75c8-581d-4079-abbf-a54a2882cc98 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:26.938826 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc75c8-581d-4079-abbf-a54a2882cc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:26.939106 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc75c8-581d-4079-abbf-a54a2882cc98 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:26.939829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc75c8-581d-4079-abbf-a54a2882cc98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:26.941424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc75c8-581d-4079-abbf-a54a2882cc98 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:26.951535 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1390/2760] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:26.955202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254336ed-832e-427d-9785-f9846e0dc3ed admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:26.964868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254336ed-832e-427d-9785-f9846e0dc3ed admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:26.969419 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fbb2c-92d6-4b08-8143-d26419942cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:26.970061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fbb2c-92d6-4b08-8143-d26419942cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:26.970341 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fbb2c-92d6-4b08-8143-d26419942cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:26.970614 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fbb2c-92d6-4b08-8143-d26419942cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:26.971357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fbb2c-92d6-4b08-8143-d26419942cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:26.973462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fbb2c-92d6-4b08-8143-d26419942cc8 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:26.984921 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1371/2761] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:26.989511 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc75c8-581d-4079-abbf-a54a2882cc98 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:26.999202 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc75c8-581d-4079-abbf-a54a2882cc98 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:27.003790 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb223dc-6d72-496f-b06f-f802bb5972c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:27.004640 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb223dc-6d72-496f-b06f-f802bb5972c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:27.004927 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb223dc-6d72-496f-b06f-f802bb5972c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:27.005207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb223dc-6d72-496f-b06f-f802bb5972c2 admin admin] PATH_INFO: `/v3/projects/835322ed5c8d47b5a078b034d38b85b2/users/6b5751d712c0404faee3c5cc144ac68a/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:27.019076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb223dc-6d72-496f-b06f-f802bb5972c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=835322ed5c8d47b5a078b034d38b85b2, user_id=6b5751d712c0404faee3c5cc144ac68a, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:27.021721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb223dc-6d72-496f-b06f-f802bb5972c2 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:27.034329 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1391/2762] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:26 2022] PUT /identity/v3/projects/835322ed5c8d47b5a078b034d38b85b2/users/6b5751d712c0404faee3c5cc144ac68a/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:27.044519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6276dcf-038c-4391-8cea-b9802454a562 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:27.044808 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6276dcf-038c-4391-8cea-b9802454a562 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:27.045093 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6276dcf-038c-4391-8cea-b9802454a562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:27.071058 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-c6276dcf-038c-4391-8cea-b9802454a562 None None] MFA Rules not processed for user `6b5751d712c0404faee3c5cc144ac68a`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:27.072268 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c6276dcf-038c-4391-8cea-b9802454a562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:27.137709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1372/2763] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:27 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:27.142352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb223dc-6d72-496f-b06f-f802bb5972c2 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:27.203022 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb223dc-6d72-496f-b06f-f802bb5972c2 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:27.208357 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9847f-05cf-4d9d-aaca-b99a3f6c65bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:27.209002 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9847f-05cf-4d9d-aaca-b99a3f6c65bb admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:27.209284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9847f-05cf-4d9d-aaca-b99a3f6c65bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:27.209562 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9847f-05cf-4d9d-aaca-b99a3f6c65bb admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:27.210250 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9847f-05cf-4d9d-aaca-b99a3f6c65bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:27.212180 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9847f-05cf-4d9d-aaca-b99a3f6c65bb admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:27.236110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1392/2764] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:02:27 2022] POST /identity/v3/projects => generated 405 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:27.240483 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6276dcf-038c-4391-8cea-b9802454a562 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:27.251005 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6276dcf-038c-4391-8cea-b9802454a562 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:27.304933 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebff7a-936d-4baf-bc94-5df8a9f0ad4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:27.305596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebff7a-936d-4baf-bc94-5df8a9f0ad4a admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:27.305881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebff7a-936d-4baf-bc94-5df8a9f0ad4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:27.306161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebff7a-936d-4baf-bc94-5df8a9f0ad4a admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:27.306871 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebff7a-936d-4baf-bc94-5df8a9f0ad4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:27.308780 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebff7a-936d-4baf-bc94-5df8a9f0ad4a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:27.328788 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1373/2765] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:02:27 2022] POST /identity/v3/users => generated 309 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:27.332685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9847f-05cf-4d9d-aaca-b99a3f6c65bb admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:27.342814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9847f-05cf-4d9d-aaca-b99a3f6c65bb admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:27.347433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9902a-78a7-4c53-9f7f-24c558f1f2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:27.348079 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9902a-78a7-4c53-9f7f-24c558f1f2de admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:27.348361 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9902a-78a7-4c53-9f7f-24c558f1f2de admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:27.348639 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9902a-78a7-4c53-9f7f-24c558f1f2de admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:27.349364 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9902a-78a7-4c53-9f7f-24c558f1f2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:27.350985 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9902a-78a7-4c53-9f7f-24c558f1f2de admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:27.361371 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1393/2766] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:27.365733 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebff7a-936d-4baf-bc94-5df8a9f0ad4a admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:27.375310 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebff7a-936d-4baf-bc94-5df8a9f0ad4a admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:27.380441 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b826bdf-52cd-4717-8c18-abd43d561a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:27.381289 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b826bdf-52cd-4717-8c18-abd43d561a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:27.381570 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b826bdf-52cd-4717-8c18-abd43d561a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:27.381844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b826bdf-52cd-4717-8c18-abd43d561a2d admin admin] PATH_INFO: `/v3/projects/f3f7291e85a34973a5a18aafd5eb232f/users/72b4b46c0fd84536b3ecfeb60dc31544/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:27.395838 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b826bdf-52cd-4717-8c18-abd43d561a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f7291e85a34973a5a18aafd5eb232f, user_id=72b4b46c0fd84536b3ecfeb60dc31544, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:27.398419 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b826bdf-52cd-4717-8c18-abd43d561a2d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:27.410717 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1374/2767] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:27 2022] PUT /identity/v3/projects/f3f7291e85a34973a5a18aafd5eb232f/users/72b4b46c0fd84536b3ecfeb60dc31544/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:27.414512 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9902a-78a7-4c53-9f7f-24c558f1f2de admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:27.424334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9902a-78a7-4c53-9f7f-24c558f1f2de admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:27.479963 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1400f0a-b0f6-41b8-a5d5-3976fbdd4556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:27.480888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1400f0a-b0f6-41b8-a5d5-3976fbdd4556 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:27.481171 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1400f0a-b0f6-41b8-a5d5-3976fbdd4556 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:27.481446 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1400f0a-b0f6-41b8-a5d5-3976fbdd4556 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:27.482155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1400f0a-b0f6-41b8-a5d5-3976fbdd4556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:27.483781 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1400f0a-b0f6-41b8-a5d5-3976fbdd4556 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:27.493709 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1394/2768] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:27.497815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b826bdf-52cd-4717-8c18-abd43d561a2d admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:27.508690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b826bdf-52cd-4717-8c18-abd43d561a2d admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:27.513385 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b1697-1749-4c8f-a057-b4fb22c4357a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:27.514256 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b1697-1749-4c8f-a057-b4fb22c4357a admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:27.514543 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b1697-1749-4c8f-a057-b4fb22c4357a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:27.514882 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b1697-1749-4c8f-a057-b4fb22c4357a admin admin] PATH_INFO: `/v3/projects/f3f7291e85a34973a5a18aafd5eb232f/users/72b4b46c0fd84536b3ecfeb60dc31544/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:27.516830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b1697-1749-4c8f-a057-b4fb22c4357a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f7291e85a34973a5a18aafd5eb232f, user_id=72b4b46c0fd84536b3ecfeb60dc31544, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:27.519503 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b1697-1749-4c8f-a057-b4fb22c4357a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:27.529479 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1375/2769] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:27 2022] PUT /identity/v3/projects/f3f7291e85a34973a5a18aafd5eb232f/users/72b4b46c0fd84536b3ecfeb60dc31544/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:27.539178 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8a4e6-7c57-41e2-a320-7e98b92fc53a None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:27.539497 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8a4e6-7c57-41e2-a320-7e98b92fc53a None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:27.539799 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8a4e6-7c57-41e2-a320-7e98b92fc53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:27.565150 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e5c8a4e6-7c57-41e2-a320-7e98b92fc53a None None] MFA Rules not processed for user `72b4b46c0fd84536b3ecfeb60dc31544`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:27.567041 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e5c8a4e6-7c57-41e2-a320-7e98b92fc53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:27.632132 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1395/2770] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:27 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 22:02:27.642799 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b1697-1749-4c8f-a057-b4fb22c4357a admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:27.644518 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b17b1697-1749-4c8f-a057-b4fb22c4357a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:27.700918 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b1697-1749-4c8f-a057-b4fb22c4357a admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:27.704618 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad3483-bea5-4eba-bd01-d66c7e9b22d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:27.705257 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad3483-bea5-4eba-bd01-d66c7e9b22d6 service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:27.705545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad3483-bea5-4eba-bd01-d66c7e9b22d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:27.705824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad3483-bea5-4eba-bd01-d66c7e9b22d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:27.714613 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad3483-bea5-4eba-bd01-d66c7e9b22d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:27.716082 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad3483-bea5-4eba-bd01-d66c7e9b22d6 service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:27.725842 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1376/2771] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:27 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:02:27.866191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8a4e6-7c57-41e2-a320-7e98b92fc53a None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:27.875623 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8a4e6-7c57-41e2-a320-7e98b92fc53a None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:27.879241 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae27c0-2e96-4814-8376-5030ec807392 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:27.879876 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae27c0-2e96-4814-8376-5030ec807392 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:27.880158 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae27c0-2e96-4814-8376-5030ec807392 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:27.880435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae27c0-2e96-4814-8376-5030ec807392 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:27.935112 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae27c0-2e96-4814-8376-5030ec807392 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:27.936725 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae27c0-2e96-4814-8376-5030ec807392 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:27.945493 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1396/2772] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:27 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:02:35.148356 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad3483-bea5-4eba-bd01-d66c7e9b22d6 service cinder] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:35.207693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad3483-bea5-4eba-bd01-d66c7e9b22d6 service cinder] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:35.212175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10cd81-3e58-49d9-9a73-a5ee6036d12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:35.213029 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10cd81-3e58-49d9-9a73-a5ee6036d12d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:35.213314 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10cd81-3e58-49d9-9a73-a5ee6036d12d admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:35.213591 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10cd81-3e58-49d9-9a73-a5ee6036d12d admin admin] PATH_INFO: `/v3/users/6b5751d712c0404faee3c5cc144ac68a` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:35.214810 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10cd81-3e58-49d9-9a73-a5ee6036d12d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b5751d712c0404faee3c5cc144ac68a)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:35.216729 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10cd81-3e58-49d9-9a73-a5ee6036d12d admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:35.259914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-de10cd81-3e58-49d9-9a73-a5ee6036d12d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b5751d712c0404faee3c5cc144ac68a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.271194 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-de10cd81-3e58-49d9-9a73-a5ee6036d12d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b5751d712c0404faee3c5cc144ac68a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.271857 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-de10cd81-3e58-49d9-9a73-a5ee6036d12d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b5751d712c0404faee3c5cc144ac68a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.272466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-de10cd81-3e58-49d9-9a73-a5ee6036d12d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b5751d712c0404faee3c5cc144ac68a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.279032 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-de10cd81-3e58-49d9-9a73-a5ee6036d12d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b5751d712c0404faee3c5cc144ac68a'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.291606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1377/2773] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:02:35 2022] DELETE /identity/v3/users/6b5751d712c0404faee3c5cc144ac68a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:35.296082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867f3a1-0a0b-495a-8fcc-830cf64b5015 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:35.296606 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867f3a1-0a0b-495a-8fcc-830cf64b5015 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:35.296715 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867f3a1-0a0b-495a-8fcc-830cf64b5015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:35.313814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e867f3a1-0a0b-495a-8fcc-830cf64b5015 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:35.315058 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e867f3a1-0a0b-495a-8fcc-830cf64b5015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:35.372149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1397/2774] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:35.381700 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10cd81-3e58-49d9-9a73-a5ee6036d12d admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:35.383200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-de10cd81-3e58-49d9-9a73-a5ee6036d12d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:35.437623 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10cd81-3e58-49d9-9a73-a5ee6036d12d admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:35.441199 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbec92-fe3e-48b2-b0aa-a15b3dbf947b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:35.441854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbec92-fe3e-48b2-b0aa-a15b3dbf947b service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:35.442136 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbec92-fe3e-48b2-b0aa-a15b3dbf947b service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:35.442411 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbec92-fe3e-48b2-b0aa-a15b3dbf947b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:35.451188 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbec92-fe3e-48b2-b0aa-a15b3dbf947b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:35.452606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbec92-fe3e-48b2-b0aa-a15b3dbf947b service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:35.461732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1378/2775] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:02:35.717774 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867f3a1-0a0b-495a-8fcc-830cf64b5015 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:35.719373 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e867f3a1-0a0b-495a-8fcc-830cf64b5015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:35.734645 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867f3a1-0a0b-495a-8fcc-830cf64b5015 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:35.740242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:35.741176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:35.741351 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:35.741671 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] PATH_INFO: `/v3/projects/835322ed5c8d47b5a078b034d38b85b2` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:35.743077 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=835322ed5c8d47b5a078b034d38b85b2)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:35.744984 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:35.790206 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '835322ed5c8d47b5a078b034d38b85b2', 'request_id': 'req-32c311de-5e65-4ed5-85b2-2cd92f6989c8'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.796093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '835322ed5c8d47b5a078b034d38b85b2', 'request_id': 'req-32c311de-5e65-4ed5-85b2-2cd92f6989c8'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.803225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '835322ed5c8d47b5a078b034d38b85b2', 'request_id': 'req-32c311de-5e65-4ed5-85b2-2cd92f6989c8'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.813836 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] The token cache is being invalidate because project 835322ed5c8d47b5a078b034d38b85b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:02:35.814063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.814732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.816292 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1398/2776] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:02:35 2022] DELETE /identity/v3/projects/835322ed5c8d47b5a078b034d38b85b2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:35.820244 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbec92-fe3e-48b2-b0aa-a15b3dbf947b service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:35.830061 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbec92-fe3e-48b2-b0aa-a15b3dbf947b service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:35.885293 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecd5ed-df95-46ce-8e1e-740003bc775c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:35.886158 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecd5ed-df95-46ce-8e1e-740003bc775c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:35.886442 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecd5ed-df95-46ce-8e1e-740003bc775c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:35.886745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecd5ed-df95-46ce-8e1e-740003bc775c admin admin] PATH_INFO: `/v3/users/72b4b46c0fd84536b3ecfeb60dc31544` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:35.887941 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecd5ed-df95-46ce-8e1e-740003bc775c admin admin] RBAC: Authorizing `identity:delete_user(user_id=72b4b46c0fd84536b3ecfeb60dc31544)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:35.889848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecd5ed-df95-46ce-8e1e-740003bc775c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:35.936143 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-55ecd5ed-df95-46ce-8e1e-740003bc775c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72b4b46c0fd84536b3ecfeb60dc31544'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.948229 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-55ecd5ed-df95-46ce-8e1e-740003bc775c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72b4b46c0fd84536b3ecfeb60dc31544'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.948908 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-55ecd5ed-df95-46ce-8e1e-740003bc775c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72b4b46c0fd84536b3ecfeb60dc31544'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.949605 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-55ecd5ed-df95-46ce-8e1e-740003bc775c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72b4b46c0fd84536b3ecfeb60dc31544'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.956325 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-55ecd5ed-df95-46ce-8e1e-740003bc775c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72b4b46c0fd84536b3ecfeb60dc31544'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:35.968912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1379/2777] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:02:35 2022] DELETE /identity/v3/users/72b4b46c0fd84536b3ecfeb60dc31544 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 22:02:36.227954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:36.229444 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:36.243070 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c311de-5e65-4ed5-85b2-2cd92f6989c8 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:36.298603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8498e47-f587-4166-917e-217fad63cbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:36.299487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8498e47-f587-4166-917e-217fad63cbbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:36.299767 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8498e47-f587-4166-917e-217fad63cbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:36.300069 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8498e47-f587-4166-917e-217fad63cbbf admin admin] PATH_INFO: `/v3/projects/f3f7291e85a34973a5a18aafd5eb232f` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:36.301745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8498e47-f587-4166-917e-217fad63cbbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3f7291e85a34973a5a18aafd5eb232f)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:36.303689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8498e47-f587-4166-917e-217fad63cbbf admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:36.346632 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f8498e47-f587-4166-917e-217fad63cbbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3f7291e85a34973a5a18aafd5eb232f', 'request_id': 'req-f8498e47-f587-4166-917e-217fad63cbbf'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:36.352381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f8498e47-f587-4166-917e-217fad63cbbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3f7291e85a34973a5a18aafd5eb232f', 'request_id': 'req-f8498e47-f587-4166-917e-217fad63cbbf'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:36.359829 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f8498e47-f587-4166-917e-217fad63cbbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3f7291e85a34973a5a18aafd5eb232f', 'request_id': 'req-f8498e47-f587-4166-917e-217fad63cbbf'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:36.372757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f8498e47-f587-4166-917e-217fad63cbbf admin admin] The token cache is being invalidate because project f3f7291e85a34973a5a18aafd5eb232f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:02:36.373248 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f8498e47-f587-4166-917e-217fad63cbbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:36.373885 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-f8498e47-f587-4166-917e-217fad63cbbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:36.375450 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1399/2778] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:02:36 2022] DELETE /identity/v3/projects/f3f7291e85a34973a5a18aafd5eb232f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:36.380843 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c778ea-0c5f-4857-889d-ced0d2712b2a None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:36.381315 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c778ea-0c5f-4857-889d-ced0d2712b2a None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:36.381417 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c778ea-0c5f-4857-889d-ced0d2712b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:36.398263 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-f6c778ea-0c5f-4857-889d-ced0d2712b2a None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:36.399482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f6c778ea-0c5f-4857-889d-ced0d2712b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:36.457401 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1380/2779] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:36.466189 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54d1f2-8151-474e-8810-4bf92bd66046 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:36.466491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54d1f2-8151-474e-8810-4bf92bd66046 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:36.466810 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54d1f2-8151-474e-8810-4bf92bd66046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:36.484856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-5c54d1f2-8151-474e-8810-4bf92bd66046 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:36.486050 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-5c54d1f2-8151-474e-8810-4bf92bd66046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:36.494502 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1400/2780] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:36.498732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c778ea-0c5f-4857-889d-ced0d2712b2a None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:36.508730 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c778ea-0c5f-4857-889d-ced0d2712b2a None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:36.513225 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cf535-5c3b-4033-b2d7-a32290ced7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:36.513869 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cf535-5c3b-4033-b2d7-a32290ced7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:36.514154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cf535-5c3b-4033-b2d7-a32290ced7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:36.514433 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cf535-5c3b-4033-b2d7-a32290ced7b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:36.515193 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cf535-5c3b-4033-b2d7-a32290ced7b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 22:02:36.515499 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cf535-5c3b-4033-b2d7-a32290ced7b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:36.517101 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cf535-5c3b-4033-b2d7-a32290ced7b6 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:36.528239 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1381/2781] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 22:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 22:02:36.532421 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54d1f2-8151-474e-8810-4bf92bd66046 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:36.541591 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54d1f2-8151-474e-8810-4bf92bd66046 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:36.546234 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a6daf-d32d-4b63-8179-119ca65c5dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:36.546903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a6daf-d32d-4b63-8179-119ca65c5dce admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:36.547192 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a6daf-d32d-4b63-8179-119ca65c5dce admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:36.547469 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a6daf-d32d-4b63-8179-119ca65c5dce admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:36.548157 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a6daf-d32d-4b63-8179-119ca65c5dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:36.550053 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a6daf-d32d-4b63-8179-119ca65c5dce admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:36.575027 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1401/2782] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:02:36 2022] POST /identity/v3/projects => generated 387 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:36.578863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cf535-5c3b-4033-b2d7-a32290ced7b6 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:36.588577 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cf535-5c3b-4033-b2d7-a32290ced7b6 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:36.644716 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298f11f-50f8-44d8-b31f-915601738c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:36.645306 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298f11f-50f8-44d8-b31f-915601738c1f admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:36.645684 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298f11f-50f8-44d8-b31f-915601738c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:36.645971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298f11f-50f8-44d8-b31f-915601738c1f admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:36.646654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298f11f-50f8-44d8-b31f-915601738c1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:36.648606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298f11f-50f8-44d8-b31f-915601738c1f admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:36.667320 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1382/2783] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:02:36 2022] POST /identity/v3/users => generated 294 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:36.671321 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a6daf-d32d-4b63-8179-119ca65c5dce admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:36.681954 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a6daf-d32d-4b63-8179-119ca65c5dce admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:36.687082 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27ea59-3168-46d2-a294-ec5bbe64de10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:36.687856 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ea59-3168-46d2-a294-ec5bbe64de10 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:36.688142 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ea59-3168-46d2-a294-ec5bbe64de10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:36.688420 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ea59-3168-46d2-a294-ec5bbe64de10 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:36.689149 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27ea59-3168-46d2-a294-ec5bbe64de10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:36.690798 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27ea59-3168-46d2-a294-ec5bbe64de10 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:36.701064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1402/2784] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:36.704866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298f11f-50f8-44d8-b31f-915601738c1f admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:36.714461 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298f11f-50f8-44d8-b31f-915601738c1f admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:36.719123 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ef3f2-44a5-41e6-a758-2e3b34a36736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:36.719686 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ef3f2-44a5-41e6-a758-2e3b34a36736 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:36.719966 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ef3f2-44a5-41e6-a758-2e3b34a36736 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:36.720240 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ef3f2-44a5-41e6-a758-2e3b34a36736 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:36.720962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ef3f2-44a5-41e6-a758-2e3b34a36736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:36.723095 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ef3f2-44a5-41e6-a758-2e3b34a36736 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:36.733276 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1383/2785] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:36.737286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27ea59-3168-46d2-a294-ec5bbe64de10 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:36.747868 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27ea59-3168-46d2-a294-ec5bbe64de10 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:36.752515 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72621a0e-ab5b-48c2-91fa-2b3bc9c6eb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:36.753358 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72621a0e-ab5b-48c2-91fa-2b3bc9c6eb3b admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:36.753637 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72621a0e-ab5b-48c2-91fa-2b3bc9c6eb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:36.753936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72621a0e-ab5b-48c2-91fa-2b3bc9c6eb3b admin admin] PATH_INFO: `/v3/projects/9ff32261cc16426fbcf5e913b1901bed/users/324a9b984a2146f78414e0a3963226d1/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:36.768861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72621a0e-ab5b-48c2-91fa-2b3bc9c6eb3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff32261cc16426fbcf5e913b1901bed, user_id=324a9b984a2146f78414e0a3963226d1, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:36.771628 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72621a0e-ab5b-48c2-91fa-2b3bc9c6eb3b admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:36.784544 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1403/2786] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:36 2022] PUT /identity/v3/projects/9ff32261cc16426fbcf5e913b1901bed/users/324a9b984a2146f78414e0a3963226d1/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 22:02:36.795095 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6ae36-02db-49e8-bfb9-e3242b825031 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:36.795390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6ae36-02db-49e8-bfb9-e3242b825031 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:36.795673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6ae36-02db-49e8-bfb9-e3242b825031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:36.821901 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-c4f6ae36-02db-49e8-bfb9-e3242b825031 None None] MFA Rules not processed for user `324a9b984a2146f78414e0a3963226d1`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:36.823150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c4f6ae36-02db-49e8-bfb9-e3242b825031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:36.884782 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1384/2787] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:36.889200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72621a0e-ab5b-48c2-91fa-2b3bc9c6eb3b admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:36.949060 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72621a0e-ab5b-48c2-91fa-2b3bc9c6eb3b admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:36.953835 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47844b1b-6ff6-4a93-895b-1ace28814268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:36.954478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47844b1b-6ff6-4a93-895b-1ace28814268 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:36.954787 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47844b1b-6ff6-4a93-895b-1ace28814268 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:36.955061 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47844b1b-6ff6-4a93-895b-1ace28814268 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:36.955737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47844b1b-6ff6-4a93-895b-1ace28814268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:36.957605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47844b1b-6ff6-4a93-895b-1ace28814268 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:36.982188 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1404/2788] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:02:36 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:36.986137 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6ae36-02db-49e8-bfb9-e3242b825031 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:36.996907 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6ae36-02db-49e8-bfb9-e3242b825031 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:37.049997 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f071d5a-db62-4b8a-97c7-f5965d2ab4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:37.050669 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f071d5a-db62-4b8a-97c7-f5965d2ab4ad admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:37.050986 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f071d5a-db62-4b8a-97c7-f5965d2ab4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:37.051261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f071d5a-db62-4b8a-97c7-f5965d2ab4ad admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:37.051937 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f071d5a-db62-4b8a-97c7-f5965d2ab4ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:37.053835 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f071d5a-db62-4b8a-97c7-f5965d2ab4ad admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:37.075671 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1385/2789] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:02:36 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:02:37.079539 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47844b1b-6ff6-4a93-895b-1ace28814268 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:37.090075 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47844b1b-6ff6-4a93-895b-1ace28814268 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:37.094673 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875703da-2c98-4e75-8982-20aff6324384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:37.095342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875703da-2c98-4e75-8982-20aff6324384 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:37.095622 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875703da-2c98-4e75-8982-20aff6324384 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:37.095893 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875703da-2c98-4e75-8982-20aff6324384 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:37.096607 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875703da-2c98-4e75-8982-20aff6324384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:37.098871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875703da-2c98-4e75-8982-20aff6324384 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:37.109200 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1405/2790] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:37.113002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f071d5a-db62-4b8a-97c7-f5965d2ab4ad admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:37.122612 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f071d5a-db62-4b8a-97c7-f5965d2ab4ad admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:37.127218 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca9b2c-4a95-4ccf-bc80-99ec4ac9df0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:37.128081 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca9b2c-4a95-4ccf-bc80-99ec4ac9df0a admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:37.128363 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca9b2c-4a95-4ccf-bc80-99ec4ac9df0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:37.128639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca9b2c-4a95-4ccf-bc80-99ec4ac9df0a admin admin] PATH_INFO: `/v3/projects/dbe458bd11a7476b83dca9dc06a34bc6/users/b3099972309a4f0891f30b747292f248/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:37.143038 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca9b2c-4a95-4ccf-bc80-99ec4ac9df0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbe458bd11a7476b83dca9dc06a34bc6, user_id=b3099972309a4f0891f30b747292f248, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:37.146202 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca9b2c-4a95-4ccf-bc80-99ec4ac9df0a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:37.159028 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1386/2791] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:37 2022] PUT /identity/v3/projects/dbe458bd11a7476b83dca9dc06a34bc6/users/b3099972309a4f0891f30b747292f248/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:37.163477 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875703da-2c98-4e75-8982-20aff6324384 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:37.173273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875703da-2c98-4e75-8982-20aff6324384 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:37.227213 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7c423-8e5c-44ea-bc8b-04c11d42dd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:37.227866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7c423-8e5c-44ea-bc8b-04c11d42dd6a admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:37.228143 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7c423-8e5c-44ea-bc8b-04c11d42dd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:37.228441 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7c423-8e5c-44ea-bc8b-04c11d42dd6a admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:37.229147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7c423-8e5c-44ea-bc8b-04c11d42dd6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:37.231320 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7c423-8e5c-44ea-bc8b-04c11d42dd6a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:37.240830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1406/2792] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:02:37.245125 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca9b2c-4a95-4ccf-bc80-99ec4ac9df0a admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:37.255184 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca9b2c-4a95-4ccf-bc80-99ec4ac9df0a admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:37.259754 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dffca-384c-441d-9c19-afe37faaf5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:37.260617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dffca-384c-441d-9c19-afe37faaf5fc admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:37.260899 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dffca-384c-441d-9c19-afe37faaf5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:37.261174 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dffca-384c-441d-9c19-afe37faaf5fc admin admin] PATH_INFO: `/v3/projects/dbe458bd11a7476b83dca9dc06a34bc6/users/b3099972309a4f0891f30b747292f248/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:37.263168 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423dffca-384c-441d-9c19-afe37faaf5fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbe458bd11a7476b83dca9dc06a34bc6, user_id=b3099972309a4f0891f30b747292f248, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:37.265803 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423dffca-384c-441d-9c19-afe37faaf5fc admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:37.276237 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1387/2793] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:02:37 2022] PUT /identity/v3/projects/dbe458bd11a7476b83dca9dc06a34bc6/users/b3099972309a4f0891f30b747292f248/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:37.287699 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e48151-b12b-4554-8789-252196f8532e None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:37.287990 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e48151-b12b-4554-8789-252196f8532e None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:37.288265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e48151-b12b-4554-8789-252196f8532e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:37.313309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e4e48151-b12b-4554-8789-252196f8532e None None] MFA Rules not processed for user `b3099972309a4f0891f30b747292f248`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:37.314489 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e4e48151-b12b-4554-8789-252196f8532e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:37.378361 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1407/2794] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:37.389054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dffca-384c-441d-9c19-afe37faaf5fc admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:37.390807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-423dffca-384c-441d-9c19-afe37faaf5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:37.446170 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dffca-384c-441d-9c19-afe37faaf5fc admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:37.449725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed808821-d325-406f-9e1e-7fae866d8f4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:37.450357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed808821-d325-406f-9e1e-7fae866d8f4a service cinder] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:37.450643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed808821-d325-406f-9e1e-7fae866d8f4a service cinder] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:37.450970 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed808821-d325-406f-9e1e-7fae866d8f4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:37.460241 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed808821-d325-406f-9e1e-7fae866d8f4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:37.461660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed808821-d325-406f-9e1e-7fae866d8f4a service cinder] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:37.470854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1388/2795] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:02:37.983813 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e48151-b12b-4554-8789-252196f8532e None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:38.043605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e48151-b12b-4554-8789-252196f8532e None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:38.048136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9dadf-2456-4564-a538-b69141f28df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:38.048994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9dadf-2456-4564-a538-b69141f28df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:38.049277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9dadf-2456-4564-a538-b69141f28df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:38.049555 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9dadf-2456-4564-a538-b69141f28df3 admin admin] PATH_INFO: `/v3/users/324a9b984a2146f78414e0a3963226d1` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:38.051263 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9dadf-2456-4564-a538-b69141f28df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=324a9b984a2146f78414e0a3963226d1)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:38.053168 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9dadf-2456-4564-a538-b69141f28df3 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:38.096139 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bfb9dadf-2456-4564-a538-b69141f28df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324a9b984a2146f78414e0a3963226d1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.107579 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bfb9dadf-2456-4564-a538-b69141f28df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324a9b984a2146f78414e0a3963226d1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.108232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bfb9dadf-2456-4564-a538-b69141f28df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324a9b984a2146f78414e0a3963226d1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.108842 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bfb9dadf-2456-4564-a538-b69141f28df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324a9b984a2146f78414e0a3963226d1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.116522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-bfb9dadf-2456-4564-a538-b69141f28df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324a9b984a2146f78414e0a3963226d1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.128523 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1408/2796] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:02:37 2022] DELETE /identity/v3/users/324a9b984a2146f78414e0a3963226d1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:38.133709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9749c4b-7816-4f69-8df2-f88a33ce8ef4 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:38.133999 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9749c4b-7816-4f69-8df2-f88a33ce8ef4 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:38.134281 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9749c4b-7816-4f69-8df2-f88a33ce8ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:38.151444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-b9749c4b-7816-4f69-8df2-f88a33ce8ef4 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:02:38.152654 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b9749c4b-7816-4f69-8df2-f88a33ce8ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:38.209065 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1389/2797] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:02:38.218544 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9dadf-2456-4564-a538-b69141f28df3 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:38.220018 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-bfb9dadf-2456-4564-a538-b69141f28df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:38.274595 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9dadf-2456-4564-a538-b69141f28df3 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:38.278172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e4b3a-f1dc-48f7-9067-ea49871f4329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:38.278832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e4b3a-f1dc-48f7-9067-ea49871f4329 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:38.279114 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e4b3a-f1dc-48f7-9067-ea49871f4329 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:38.279385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e4b3a-f1dc-48f7-9067-ea49871f4329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:38.288685 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e4b3a-f1dc-48f7-9067-ea49871f4329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:38.290102 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e4b3a-f1dc-48f7-9067-ea49871f4329 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:38.299179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1409/2798] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:02:38.555158 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9749c4b-7816-4f69-8df2-f88a33ce8ef4 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:38.556644 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-b9749c4b-7816-4f69-8df2-f88a33ce8ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:38.568466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9749c4b-7816-4f69-8df2-f88a33ce8ef4 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:38.573160 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:38.574020 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:38.574301 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:38.574577 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] PATH_INFO: `/v3/projects/9ff32261cc16426fbcf5e913b1901bed` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:38.576323 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ff32261cc16426fbcf5e913b1901bed)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:38.578300 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:38.625925 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ff32261cc16426fbcf5e913b1901bed', 'request_id': 'req-96a11243-91fb-4590-876c-21eb9a4de617'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.632105 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ff32261cc16426fbcf5e913b1901bed', 'request_id': 'req-96a11243-91fb-4590-876c-21eb9a4de617'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.639493 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ff32261cc16426fbcf5e913b1901bed', 'request_id': 'req-96a11243-91fb-4590-876c-21eb9a4de617'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.650956 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] The token cache is being invalidate because project 9ff32261cc16426fbcf5e913b1901bed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:02:38.651305 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.651965 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.653573 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1390/2799] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:02:38 2022] DELETE /identity/v3/projects/9ff32261cc16426fbcf5e913b1901bed => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:38.657451 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e4b3a-f1dc-48f7-9067-ea49871f4329 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:38.667691 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e4b3a-f1dc-48f7-9067-ea49871f4329 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:38.724968 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a3dfe-153a-4975-afe6-7f52de1825ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:38.725831 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a3dfe-153a-4975-afe6-7f52de1825ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:38.726116 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a3dfe-153a-4975-afe6-7f52de1825ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:38.726392 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a3dfe-153a-4975-afe6-7f52de1825ec admin admin] PATH_INFO: `/v3/users/b3099972309a4f0891f30b747292f248` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:38.727607 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a3dfe-153a-4975-afe6-7f52de1825ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3099972309a4f0891f30b747292f248)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:38.729537 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a3dfe-153a-4975-afe6-7f52de1825ec admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:38.773001 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9b8a3dfe-153a-4975-afe6-7f52de1825ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3099972309a4f0891f30b747292f248'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.784225 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9b8a3dfe-153a-4975-afe6-7f52de1825ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3099972309a4f0891f30b747292f248'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.784881 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9b8a3dfe-153a-4975-afe6-7f52de1825ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3099972309a4f0891f30b747292f248'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.786072 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9b8a3dfe-153a-4975-afe6-7f52de1825ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3099972309a4f0891f30b747292f248'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.792670 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-9b8a3dfe-153a-4975-afe6-7f52de1825ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3099972309a4f0891f30b747292f248'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:38.804294 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1410/2800] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:02:38 2022] DELETE /identity/v3/users/b3099972309a4f0891f30b747292f248 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 22:02:39.073447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:39.074923 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:39.088615 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a11243-91fb-4590-876c-21eb9a4de617 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:39.148502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:39.149375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:39.149660 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:39.149935 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] PATH_INFO: `/v3/projects/dbe458bd11a7476b83dca9dc06a34bc6` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:39.151206 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbe458bd11a7476b83dca9dc06a34bc6)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:39.153120 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:39.199971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbe458bd11a7476b83dca9dc06a34bc6', 'request_id': 'req-9352101c-138c-4f08-9daf-f89c30cd359c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:39.206073 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbe458bd11a7476b83dca9dc06a34bc6', 'request_id': 'req-9352101c-138c-4f08-9daf-f89c30cd359c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:39.213299 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbe458bd11a7476b83dca9dc06a34bc6', 'request_id': 'req-9352101c-138c-4f08-9daf-f89c30cd359c'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:39.225852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] The token cache is being invalidate because project dbe458bd11a7476b83dca9dc06a34bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:02:39.226141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:39.226879 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:02:39.228500 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1391/2801] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:02:39 2022] DELETE /identity/v3/projects/dbe458bd11a7476b83dca9dc06a34bc6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:02:56.171644 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a3dfe-153a-4975-afe6-7f52de1825ec admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:56.175689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-9b8a3dfe-153a-4975-afe6-7f52de1825ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:56.289408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a3dfe-153a-4975-afe6-7f52de1825ec admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:56.366915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b9b4f-4f60-411b-a7a9-68eaebe361c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:56.368129 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b9b4f-4f60-411b-a7a9-68eaebe361c3 service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:56.368748 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b9b4f-4f60-411b-a7a9-68eaebe361c3 service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:56.369246 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b9b4f-4f60-411b-a7a9-68eaebe361c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:56.371179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-751b9b4f-4f60-411b-a7a9-68eaebe361c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:56.387162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b9b4f-4f60-411b-a7a9-68eaebe361c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:56.388798 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b9b4f-4f60-411b-a7a9-68eaebe361c3 service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:56.400220 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1411/2802] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:02:56.598056 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:02:56.600690 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:56.664428 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352101c-138c-4f08-9daf-f89c30cd359c admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:02:56.668539 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e92b8-fe46-43e6-a9c4-bfbd7f814635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:02:56.669313 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e92b8-fe46-43e6-a9c4-bfbd7f814635 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:02:56.669732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e92b8-fe46-43e6-a9c4-bfbd7f814635 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:02:56.670250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e92b8-fe46-43e6-a9c4-bfbd7f814635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:02:56.672199 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-779e92b8-fe46-43e6-a9c4-bfbd7f814635 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:02:56.683681 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e92b8-fe46-43e6-a9c4-bfbd7f814635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:02:56.685366 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e92b8-fe46-43e6-a9c4-bfbd7f814635 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:02:56.696015 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1392/2803] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:02:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:03:04.309367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b9b4f-4f60-411b-a7a9-68eaebe361c3 service nova] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:04.310883 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-751b9b4f-4f60-411b-a7a9-68eaebe361c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:04.380773 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b9b4f-4f60-411b-a7a9-68eaebe361c3 service nova] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:04.385636 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d117e6-0076-411e-b545-a3c0aca7379d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:04.386495 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d117e6-0076-411e-b545-a3c0aca7379d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:04.386821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d117e6-0076-411e-b545-a3c0aca7379d admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:04.387103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d117e6-0076-411e-b545-a3c0aca7379d admin admin] PATH_INFO: `/v3/users/dddbe23734ee4cee866d58e3314007ea` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:04.388286 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d117e6-0076-411e-b545-a3c0aca7379d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dddbe23734ee4cee866d58e3314007ea)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:04.390238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d117e6-0076-411e-b545-a3c0aca7379d admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:04.434098 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-20d117e6-0076-411e-b545-a3c0aca7379d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dddbe23734ee4cee866d58e3314007ea'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:04.445222 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-20d117e6-0076-411e-b545-a3c0aca7379d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dddbe23734ee4cee866d58e3314007ea'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:04.445882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-20d117e6-0076-411e-b545-a3c0aca7379d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dddbe23734ee4cee866d58e3314007ea'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:04.446487 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-20d117e6-0076-411e-b545-a3c0aca7379d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dddbe23734ee4cee866d58e3314007ea'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:04.452983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-20d117e6-0076-411e-b545-a3c0aca7379d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dddbe23734ee4cee866d58e3314007ea'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:04.464830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1412/2804] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:03:04 2022] DELETE /identity/v3/users/dddbe23734ee4cee866d58e3314007ea => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:03:04.638423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e92b8-fe46-43e6-a9c4-bfbd7f814635 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:04.651337 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e92b8-fe46-43e6-a9c4-bfbd7f814635 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:04.708565 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb4b0a-6928-4b06-9797-1f47218cf751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:04.709987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb4b0a-6928-4b06-9797-1f47218cf751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:04.710275 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb4b0a-6928-4b06-9797-1f47218cf751 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:04.710558 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb4b0a-6928-4b06-9797-1f47218cf751 admin admin] PATH_INFO: `/v3/projects/9e5d643885114a45b305ac13bef897d3` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:04.711787 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb4b0a-6928-4b06-9797-1f47218cf751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e5d643885114a45b305ac13bef897d3)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:04.713705 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb4b0a-6928-4b06-9797-1f47218cf751 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:04.760359 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-49fb4b0a-6928-4b06-9797-1f47218cf751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e5d643885114a45b305ac13bef897d3', 'request_id': 'req-49fb4b0a-6928-4b06-9797-1f47218cf751'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:04.766410 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-49fb4b0a-6928-4b06-9797-1f47218cf751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e5d643885114a45b305ac13bef897d3', 'request_id': 'req-49fb4b0a-6928-4b06-9797-1f47218cf751'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:04.774071 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-49fb4b0a-6928-4b06-9797-1f47218cf751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e5d643885114a45b305ac13bef897d3', 'request_id': 'req-49fb4b0a-6928-4b06-9797-1f47218cf751'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:04.784219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-49fb4b0a-6928-4b06-9797-1f47218cf751 admin admin] The token cache is being invalidate because project 9e5d643885114a45b305ac13bef897d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:03:04.784506 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-49fb4b0a-6928-4b06-9797-1f47218cf751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:04.785131 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-49fb4b0a-6928-4b06-9797-1f47218cf751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:04.787466 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1393/2805] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:03:04 2022] DELETE /identity/v3/projects/9e5d643885114a45b305ac13bef897d3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 22:03:04.792243 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b2469-fb6c-4c45-a51f-4c04886d30af None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:04.792605 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b2469-fb6c-4c45-a51f-4c04886d30af None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:04.792958 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b2469-fb6c-4c45-a51f-4c04886d30af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:04.810545 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-092b2469-fb6c-4c45-a51f-4c04886d30af None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:03:04.811830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-092b2469-fb6c-4c45-a51f-4c04886d30af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:04.872339 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1413/2806] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:03:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 22:03:04.880577 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c517742-e520-409f-bde3-8c3dccc76f59 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:04.880874 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c517742-e520-409f-bde3-8c3dccc76f59 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:04.881158 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c517742-e520-409f-bde3-8c3dccc76f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:04.899476 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-9c517742-e520-409f-bde3-8c3dccc76f59 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:03:04.900695 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-9c517742-e520-409f-bde3-8c3dccc76f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:04.908525 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1394/2807] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:03:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:03:04.913089 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b2469-fb6c-4c45-a51f-4c04886d30af None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:04.923242 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b2469-fb6c-4c45-a51f-4c04886d30af None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:04.928284 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7890c3b-3c6f-478d-b685-63330ed5bfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:04.928983 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7890c3b-3c6f-478d-b685-63330ed5bfe9 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:04.929233 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7890c3b-3c6f-478d-b685-63330ed5bfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:04.929513 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7890c3b-3c6f-478d-b685-63330ed5bfe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:04.933811 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7890c3b-3c6f-478d-b685-63330ed5bfe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 22:03:04.934135 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7890c3b-3c6f-478d-b685-63330ed5bfe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:04.935815 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7890c3b-3c6f-478d-b685-63330ed5bfe9 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:04.948236 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1414/2808] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 22:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 22:03:04.952596 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c517742-e520-409f-bde3-8c3dccc76f59 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:04.962260 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c517742-e520-409f-bde3-8c3dccc76f59 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:04.966952 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca922a3b-afe4-42bd-977f-9015e430f2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:04.967603 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca922a3b-afe4-42bd-977f-9015e430f2f8 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:04.967888 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca922a3b-afe4-42bd-977f-9015e430f2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:04.968164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca922a3b-afe4-42bd-977f-9015e430f2f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:04.968885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca922a3b-afe4-42bd-977f-9015e430f2f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:04.970844 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca922a3b-afe4-42bd-977f-9015e430f2f8 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:04.996272 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1395/2809] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:03:04 2022] POST /identity/v3/projects => generated 387 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:03:05.000312 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7890c3b-3c6f-478d-b685-63330ed5bfe9 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:05.010197 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7890c3b-3c6f-478d-b685-63330ed5bfe9 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:05.063761 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d12184-c4ad-4862-86f8-d8ee0200bd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:05.064347 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d12184-c4ad-4862-86f8-d8ee0200bd16 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:05.064633 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d12184-c4ad-4862-86f8-d8ee0200bd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:05.064909 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d12184-c4ad-4862-86f8-d8ee0200bd16 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:05.065653 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d12184-c4ad-4862-86f8-d8ee0200bd16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:05.067744 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d12184-c4ad-4862-86f8-d8ee0200bd16 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:05.086692 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1415/2810] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:03:04 2022] POST /identity/v3/users => generated 294 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 22:03:05.090677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca922a3b-afe4-42bd-977f-9015e430f2f8 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:05.101355 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca922a3b-afe4-42bd-977f-9015e430f2f8 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:05.106180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e499f-3ed6-4de1-b10f-8529204047de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:05.106871 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e499f-3ed6-4de1-b10f-8529204047de admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:05.107161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e499f-3ed6-4de1-b10f-8529204047de admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:05.107444 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e499f-3ed6-4de1-b10f-8529204047de admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:05.108167 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e499f-3ed6-4de1-b10f-8529204047de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:05.109769 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e499f-3ed6-4de1-b10f-8529204047de admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:05.119813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1396/2811] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:03:05.123505 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d12184-c4ad-4862-86f8-d8ee0200bd16 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:05.133736 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d12184-c4ad-4862-86f8-d8ee0200bd16 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:05.138310 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0d9c6-f587-4857-896f-2c41bf932e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:05.138986 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0d9c6-f587-4857-896f-2c41bf932e3c admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:05.139271 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0d9c6-f587-4857-896f-2c41bf932e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:05.139549 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0d9c6-f587-4857-896f-2c41bf932e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:05.140265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0d9c6-f587-4857-896f-2c41bf932e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:05.141852 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0d9c6-f587-4857-896f-2c41bf932e3c admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:05.155108 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1416/2812] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:03:05.156176 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e499f-3ed6-4de1-b10f-8529204047de admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:05.165973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e499f-3ed6-4de1-b10f-8529204047de admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:05.170559 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e69928-422b-4995-bdf8-fa5b8199f750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:05.171462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e69928-422b-4995-bdf8-fa5b8199f750 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:05.171748 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e69928-422b-4995-bdf8-fa5b8199f750 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:05.172027 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e69928-422b-4995-bdf8-fa5b8199f750 admin admin] PATH_INFO: `/v3/projects/036be95cb0df4b8d9d612a025ceca00d/users/a26e795c37934213831c5a82a80c3cbe/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:05.185763 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e69928-422b-4995-bdf8-fa5b8199f750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=036be95cb0df4b8d9d612a025ceca00d, user_id=a26e795c37934213831c5a82a80c3cbe, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:05.188489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e69928-422b-4995-bdf8-fa5b8199f750 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:05.201422 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1397/2813] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:03:05 2022] PUT /identity/v3/projects/036be95cb0df4b8d9d612a025ceca00d/users/a26e795c37934213831c5a82a80c3cbe/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:03:05.212097 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5cb07-10e5-4953-9559-9fb29c78b710 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:05.212385 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5cb07-10e5-4953-9559-9fb29c78b710 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:05.212659 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5cb07-10e5-4953-9559-9fb29c78b710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:05.238419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-dfb5cb07-10e5-4953-9559-9fb29c78b710 None None] MFA Rules not processed for user `a26e795c37934213831c5a82a80c3cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:03:05.239634 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-dfb5cb07-10e5-4953-9559-9fb29c78b710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:05.303393 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1417/2814] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:03:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:03:05.307810 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e69928-422b-4995-bdf8-fa5b8199f750 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:05.367652 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e69928-422b-4995-bdf8-fa5b8199f750 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:05.372169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48a1cf-7750-4813-92dc-406f3fb7389c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:05.372813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48a1cf-7750-4813-92dc-406f3fb7389c admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:05.373095 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48a1cf-7750-4813-92dc-406f3fb7389c admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:05.373369 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48a1cf-7750-4813-92dc-406f3fb7389c admin admin] PATH_INFO: `/v3/projects` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:05.374054 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48a1cf-7750-4813-92dc-406f3fb7389c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:05.375979 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48a1cf-7750-4813-92dc-406f3fb7389c admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:05.401366 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1398/2815] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:03:05 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:03:05.405375 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5cb07-10e5-4953-9559-9fb29c78b710 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:05.415831 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5cb07-10e5-4953-9559-9fb29c78b710 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:05.469733 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1664ff5-3490-4fb4-b767-d1c610dde129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:05.470389 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1664ff5-3490-4fb4-b767-d1c610dde129 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:05.470674 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1664ff5-3490-4fb4-b767-d1c610dde129 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:05.470979 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1664ff5-3490-4fb4-b767-d1c610dde129 admin admin] PATH_INFO: `/v3/users` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:05.471668 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1664ff5-3490-4fb4-b767-d1c610dde129 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:05.473573 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1664ff5-3490-4fb4-b767-d1c610dde129 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:05.495005 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1418/2816] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:03:05 2022] POST /identity/v3/users => generated 298 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 22:03:05.498637 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48a1cf-7750-4813-92dc-406f3fb7389c admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:05.509559 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48a1cf-7750-4813-92dc-406f3fb7389c admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:05.514183 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0c82f-b7d2-4b50-9bef-032eaa995260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:05.514866 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0c82f-b7d2-4b50-9bef-032eaa995260 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:05.515158 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0c82f-b7d2-4b50-9bef-032eaa995260 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:05.515436 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0c82f-b7d2-4b50-9bef-032eaa995260 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:05.516159 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0c82f-b7d2-4b50-9bef-032eaa995260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:05.517751 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0c82f-b7d2-4b50-9bef-032eaa995260 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:05.527900 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1399/2817] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:03:05.531758 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1664ff5-3490-4fb4-b767-d1c610dde129 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:05.541469 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1664ff5-3490-4fb4-b767-d1c610dde129 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:05.546058 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c42c31-9fc5-479b-acf0-e49c046e738e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:05.546948 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c42c31-9fc5-479b-acf0-e49c046e738e admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:05.547232 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c42c31-9fc5-479b-acf0-e49c046e738e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:05.547510 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c42c31-9fc5-479b-acf0-e49c046e738e admin admin] PATH_INFO: `/v3/projects/a2a28a6ea7a74a26a6fde5e7ab64d709/users/0ca70e782df944d888377c098594f081/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:05.562265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c42c31-9fc5-479b-acf0-e49c046e738e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a28a6ea7a74a26a6fde5e7ab64d709, user_id=0ca70e782df944d888377c098594f081, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:05.564938 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c42c31-9fc5-479b-acf0-e49c046e738e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:05.581043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1419/2818] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:03:05 2022] PUT /identity/v3/projects/a2a28a6ea7a74a26a6fde5e7ab64d709/users/0ca70e782df944d888377c098594f081/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:03:05.585014 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0c82f-b7d2-4b50-9bef-032eaa995260 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:05.594662 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0c82f-b7d2-4b50-9bef-032eaa995260 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:05.648860 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c8a62-b6d6-4ecd-9b61-b7a28b847066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:05.649490 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c8a62-b6d6-4ecd-9b61-b7a28b847066 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:05.649705 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c8a62-b6d6-4ecd-9b61-b7a28b847066 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:05.649978 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c8a62-b6d6-4ecd-9b61-b7a28b847066 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:05.650684 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c8a62-b6d6-4ecd-9b61-b7a28b847066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:05.652298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c8a62-b6d6-4ecd-9b61-b7a28b847066 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:05.662166 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1400/2819] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:03:05.666410 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c42c31-9fc5-479b-acf0-e49c046e738e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:05.676721 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c42c31-9fc5-479b-acf0-e49c046e738e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:05.681807 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8e4b1-9387-4679-9d1c-9012693bb0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:05.682669 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8e4b1-9387-4679-9d1c-9012693bb0a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:05.682980 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8e4b1-9387-4679-9d1c-9012693bb0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:05.683257 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8e4b1-9387-4679-9d1c-9012693bb0a7 admin admin] PATH_INFO: `/v3/projects/a2a28a6ea7a74a26a6fde5e7ab64d709/users/0ca70e782df944d888377c098594f081/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:05.685208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8e4b1-9387-4679-9d1c-9012693bb0a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a28a6ea7a74a26a6fde5e7ab64d709, user_id=0ca70e782df944d888377c098594f081, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:05.687882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8e4b1-9387-4679-9d1c-9012693bb0a7 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:05.698333 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1420/2820] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:03:05 2022] PUT /identity/v3/projects/a2a28a6ea7a74a26a6fde5e7ab64d709/users/0ca70e782df944d888377c098594f081/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 22:03:05.708270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e184adb-9ad7-4cde-9c52-c1a55bcedc15 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:05.708563 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e184adb-9ad7-4cde-9c52-c1a55bcedc15 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:05.709414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e184adb-9ad7-4cde-9c52-c1a55bcedc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:05.735655 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5e184adb-9ad7-4cde-9c52-c1a55bcedc15 None None] MFA Rules not processed for user `0ca70e782df944d888377c098594f081`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:03:05.736872 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5e184adb-9ad7-4cde-9c52-c1a55bcedc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:05.801684 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1401/2821] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:03:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 22:03:05.812823 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8e4b1-9387-4679-9d1c-9012693bb0a7 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:05.814491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-8cc8e4b1-9387-4679-9d1c-9012693bb0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:05.870433 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8e4b1-9387-4679-9d1c-9012693bb0a7 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:05.874068 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735c098-87a8-4635-9d84-02bb8de30c9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:05.874715 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735c098-87a8-4635-9d84-02bb8de30c9f service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:05.875026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735c098-87a8-4635-9d84-02bb8de30c9f service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:05.875306 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735c098-87a8-4635-9d84-02bb8de30c9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:05.884922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735c098-87a8-4635-9d84-02bb8de30c9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:05.886362 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735c098-87a8-4635-9d84-02bb8de30c9f service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:05.896020 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1421/2822] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:03:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:03:05.959636 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e184adb-9ad7-4cde-9c52-c1a55bcedc15 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:06.019265 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e184adb-9ad7-4cde-9c52-c1a55bcedc15 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:06.023822 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45ecae-3260-4a0e-b28f-53e16a433d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:06.024684 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45ecae-3260-4a0e-b28f-53e16a433d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:06.024971 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45ecae-3260-4a0e-b28f-53e16a433d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:06.025249 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45ecae-3260-4a0e-b28f-53e16a433d16 admin admin] PATH_INFO: `/v3/users/a26e795c37934213831c5a82a80c3cbe` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:06.026445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45ecae-3260-4a0e-b28f-53e16a433d16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a26e795c37934213831c5a82a80c3cbe)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:06.028473 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45ecae-3260-4a0e-b28f-53e16a433d16 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:06.073508 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9d45ecae-3260-4a0e-b28f-53e16a433d16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a26e795c37934213831c5a82a80c3cbe'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.084627 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9d45ecae-3260-4a0e-b28f-53e16a433d16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a26e795c37934213831c5a82a80c3cbe'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.085277 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9d45ecae-3260-4a0e-b28f-53e16a433d16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a26e795c37934213831c5a82a80c3cbe'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.085884 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9d45ecae-3260-4a0e-b28f-53e16a433d16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a26e795c37934213831c5a82a80c3cbe'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.092468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-9d45ecae-3260-4a0e-b28f-53e16a433d16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a26e795c37934213831c5a82a80c3cbe'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.104739 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1402/2823] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:03:05 2022] DELETE /identity/v3/users/a26e795c37934213831c5a82a80c3cbe => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:03:06.109455 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b27f51-8c93-4ea9-b7a3-cbdb49c5f7fc None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:06.109747 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b27f51-8c93-4ea9-b7a3-cbdb49c5f7fc None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:06.110028 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b27f51-8c93-4ea9-b7a3-cbdb49c5f7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:06.127689 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-53b27f51-8c93-4ea9-b7a3-cbdb49c5f7fc None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:03:06.128871 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-53b27f51-8c93-4ea9-b7a3-cbdb49c5f7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:06.184322 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1422/2824] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:03:06.193968 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45ecae-3260-4a0e-b28f-53e16a433d16 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:06.195426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-9d45ecae-3260-4a0e-b28f-53e16a433d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:06.252201 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45ecae-3260-4a0e-b28f-53e16a433d16 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:06.255821 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb22cec-460b-4437-976b-5929240d9e94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:06.256454 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb22cec-460b-4437-976b-5929240d9e94 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:06.256735 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb22cec-460b-4437-976b-5929240d9e94 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:06.257009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb22cec-460b-4437-976b-5929240d9e94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:06.265741 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb22cec-460b-4437-976b-5929240d9e94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:06.267212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb22cec-460b-4437-976b-5929240d9e94 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:06.277930 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1403/2825] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:03:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:03:06.533015 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b27f51-8c93-4ea9-b7a3-cbdb49c5f7fc None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:06.534459 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-53b27f51-8c93-4ea9-b7a3-cbdb49c5f7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:06.546966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b27f51-8c93-4ea9-b7a3-cbdb49c5f7fc None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:06.551863 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:06.553334 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:06.553699 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:06.554043 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] PATH_INFO: `/v3/projects/036be95cb0df4b8d9d612a025ceca00d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:06.555389 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=036be95cb0df4b8d9d612a025ceca00d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:06.557688 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:06.606207 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '036be95cb0df4b8d9d612a025ceca00d', 'request_id': 'req-1493e896-d1d5-4968-8b4d-202aa8004bcf'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.612124 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '036be95cb0df4b8d9d612a025ceca00d', 'request_id': 'req-1493e896-d1d5-4968-8b4d-202aa8004bcf'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.619047 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '036be95cb0df4b8d9d612a025ceca00d', 'request_id': 'req-1493e896-d1d5-4968-8b4d-202aa8004bcf'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.628889 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] The token cache is being invalidate because project 036be95cb0df4b8d9d612a025ceca00d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:03:06.629176 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.629788 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.631860 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1423/2826] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:03:06 2022] DELETE /identity/v3/projects/036be95cb0df4b8d9d612a025ceca00d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:03:06.635809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb22cec-460b-4437-976b-5929240d9e94 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:06.645175 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb22cec-460b-4437-976b-5929240d9e94 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:06.701973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95c958-84b8-44f5-8cf3-df9b3818a97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:06.702881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c958-84b8-44f5-8cf3-df9b3818a97f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:06.703170 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c958-84b8-44f5-8cf3-df9b3818a97f admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:06.703450 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c958-84b8-44f5-8cf3-df9b3818a97f admin admin] PATH_INFO: `/v3/users/0ca70e782df944d888377c098594f081` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:06.704639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95c958-84b8-44f5-8cf3-df9b3818a97f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ca70e782df944d888377c098594f081)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:06.706576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95c958-84b8-44f5-8cf3-df9b3818a97f admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:06.751138 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-7b95c958-84b8-44f5-8cf3-df9b3818a97f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ca70e782df944d888377c098594f081'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.762985 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-7b95c958-84b8-44f5-8cf3-df9b3818a97f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ca70e782df944d888377c098594f081'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.763643 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-7b95c958-84b8-44f5-8cf3-df9b3818a97f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ca70e782df944d888377c098594f081'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.764250 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-7b95c958-84b8-44f5-8cf3-df9b3818a97f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ca70e782df944d888377c098594f081'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.770813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-7b95c958-84b8-44f5-8cf3-df9b3818a97f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ca70e782df944d888377c098594f081'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:06.783619 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1404/2827] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:03:06 2022] DELETE /identity/v3/users/0ca70e782df944d888377c098594f081 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 22:03:07.037423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:07.038891 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:07.051824 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493e896-d1d5-4968-8b4d-202aa8004bcf admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:07.107491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05568783-3425-446f-a79b-f26b134eb167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:07.108336 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05568783-3425-446f-a79b-f26b134eb167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:07.108615 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05568783-3425-446f-a79b-f26b134eb167 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:07.108888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05568783-3425-446f-a79b-f26b134eb167 admin admin] PATH_INFO: `/v3/projects/a2a28a6ea7a74a26a6fde5e7ab64d709` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:07.110044 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05568783-3425-446f-a79b-f26b134eb167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2a28a6ea7a74a26a6fde5e7ab64d709)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:07.111978 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05568783-3425-446f-a79b-f26b134eb167 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:07.156076 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-05568783-3425-446f-a79b-f26b134eb167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2a28a6ea7a74a26a6fde5e7ab64d709', 'request_id': 'req-05568783-3425-446f-a79b-f26b134eb167'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:07.162030 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-05568783-3425-446f-a79b-f26b134eb167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2a28a6ea7a74a26a6fde5e7ab64d709', 'request_id': 'req-05568783-3425-446f-a79b-f26b134eb167'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:07.169039 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-05568783-3425-446f-a79b-f26b134eb167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2a28a6ea7a74a26a6fde5e7ab64d709', 'request_id': 'req-05568783-3425-446f-a79b-f26b134eb167'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:07.179317 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-05568783-3425-446f-a79b-f26b134eb167 admin admin] The token cache is being invalidate because project a2a28a6ea7a74a26a6fde5e7ab64d709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:03:07.179603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-05568783-3425-446f-a79b-f26b134eb167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:07.180237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-05568783-3425-446f-a79b-f26b134eb167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:07.181755 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1424/2828] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:03:07 2022] DELETE /identity/v3/projects/a2a28a6ea7a74a26a6fde5e7ab64d709 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:03:07.186778 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c878b50-58c5-4796-9eb6-469a13c1b0c5 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:07.187077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c878b50-58c5-4796-9eb6-469a13c1b0c5 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:07.187364 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c878b50-58c5-4796-9eb6-469a13c1b0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:07.204973 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-1c878b50-58c5-4796-9eb6-469a13c1b0c5 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:03:07.206173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-1c878b50-58c5-4796-9eb6-469a13c1b0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:07.263824 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1405/2829] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:03:07.272607 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c77948-8036-4d83-97a9-1ee26127a8b2 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:07.272929 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c77948-8036-4d83-97a9-1ee26127a8b2 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:07.273205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c77948-8036-4d83-97a9-1ee26127a8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:07.291888 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-84c77948-8036-4d83-97a9-1ee26127a8b2 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:03:07.293094 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-84c77948-8036-4d83-97a9-1ee26127a8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:07.300951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1425/2830] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:03:07.305585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c878b50-58c5-4796-9eb6-469a13c1b0c5 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:07.315049 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c878b50-58c5-4796-9eb6-469a13c1b0c5 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:07.320380 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b8781-7076-489e-91e0-3642f2d42879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:07.321033 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b8781-7076-489e-91e0-3642f2d42879 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:07.321316 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b8781-7076-489e-91e0-3642f2d42879 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:07.321639 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b8781-7076-489e-91e0-3642f2d42879 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:07.322365 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b8781-7076-489e-91e0-3642f2d42879 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 22:03:07.322698 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b8781-7076-489e-91e0-3642f2d42879 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:07.324423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b8781-7076-489e-91e0-3642f2d42879 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:07.335674 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1406/2831] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 22:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 22:03:07.339822 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c77948-8036-4d83-97a9-1ee26127a8b2 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:07.349215 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c77948-8036-4d83-97a9-1ee26127a8b2 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:07.354352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d3000-f35b-4fb5-a59d-4ddda923f9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:07.355269 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d3000-f35b-4fb5-a59d-4ddda923f9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:07.355560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d3000-f35b-4fb5-a59d-4ddda923f9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:07.355838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d3000-f35b-4fb5-a59d-4ddda923f9c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:07.356521 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d3000-f35b-4fb5-a59d-4ddda923f9c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:07.358419 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d3000-f35b-4fb5-a59d-4ddda923f9c6 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:07.384000 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1426/2832] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:03:07 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:03:07.387991 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b8781-7076-489e-91e0-3642f2d42879 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:07.397673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b8781-7076-489e-91e0-3642f2d42879 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:07.451555 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629c215-f032-4b3e-9547-6c6efc86f1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:07.452212 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629c215-f032-4b3e-9547-6c6efc86f1bb admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:07.452502 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629c215-f032-4b3e-9547-6c6efc86f1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:07.453096 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629c215-f032-4b3e-9547-6c6efc86f1bb admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:07.453818 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629c215-f032-4b3e-9547-6c6efc86f1bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:07.455807 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629c215-f032-4b3e-9547-6c6efc86f1bb admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:07.475035 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1407/2833] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:03:07 2022] POST /identity/v3/users => generated 289 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 22:03:07.479121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d3000-f35b-4fb5-a59d-4ddda923f9c6 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:07.489282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d3000-f35b-4fb5-a59d-4ddda923f9c6 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:07.494366 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392985a-6058-4ce5-b31f-03da8afefb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:07.495105 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392985a-6058-4ce5-b31f-03da8afefb54 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:07.495388 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392985a-6058-4ce5-b31f-03da8afefb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:07.495665 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392985a-6058-4ce5-b31f-03da8afefb54 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:07.496383 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392985a-6058-4ce5-b31f-03da8afefb54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:07.497969 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392985a-6058-4ce5-b31f-03da8afefb54 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:07.507982 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1427/2834] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:03:07.511580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629c215-f032-4b3e-9547-6c6efc86f1bb admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:07.521077 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629c215-f032-4b3e-9547-6c6efc86f1bb admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:07.525540 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f22434-d9a8-4c27-8d30-fe774956d331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:07.526180 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f22434-d9a8-4c27-8d30-fe774956d331 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:07.526462 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f22434-d9a8-4c27-8d30-fe774956d331 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:07.526767 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f22434-d9a8-4c27-8d30-fe774956d331 admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:07.527488 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f22434-d9a8-4c27-8d30-fe774956d331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:07.529059 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f22434-d9a8-4c27-8d30-fe774956d331 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:07.539169 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1408/2835] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:03:07.542951 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392985a-6058-4ce5-b31f-03da8afefb54 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:07.553252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392985a-6058-4ce5-b31f-03da8afefb54 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:07.558007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32002f-e2d8-41ae-80ea-d9ab8c320162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:07.558904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32002f-e2d8-41ae-80ea-d9ab8c320162 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:07.559186 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32002f-e2d8-41ae-80ea-d9ab8c320162 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:07.559466 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32002f-e2d8-41ae-80ea-d9ab8c320162 admin admin] PATH_INFO: `/v3/projects/3faebb433d194e88931d6374768b9b9c/users/d009c3d45dba4e7284a7cab36f785263/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:07.576772 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32002f-e2d8-41ae-80ea-d9ab8c320162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3faebb433d194e88931d6374768b9b9c, user_id=d009c3d45dba4e7284a7cab36f785263, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:07.579482 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32002f-e2d8-41ae-80ea-d9ab8c320162 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:07.592273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1428/2836] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:03:07 2022] PUT /identity/v3/projects/3faebb433d194e88931d6374768b9b9c/users/d009c3d45dba4e7284a7cab36f785263/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 22:03:07.603152 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98613c-4e9b-44db-9a84-de62b3dbe2f1 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:07.603445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98613c-4e9b-44db-9a84-de62b3dbe2f1 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:07.603725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98613c-4e9b-44db-9a84-de62b3dbe2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:07.629908 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-2b98613c-4e9b-44db-9a84-de62b3dbe2f1 None None] MFA Rules not processed for user `d009c3d45dba4e7284a7cab36f785263`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:03:07.631161 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-2b98613c-4e9b-44db-9a84-de62b3dbe2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:07.692384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1409/2837] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:03:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:03:07.703442 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32002f-e2d8-41ae-80ea-d9ab8c320162 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:07.705120 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-6b32002f-e2d8-41ae-80ea-d9ab8c320162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:07.761005 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32002f-e2d8-41ae-80ea-d9ab8c320162 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:07.764714 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaefd97-0969-4610-a37c-27382e50dfde service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad57b046f3a43978d1c98bcc1a76d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:07.765352 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaefd97-0969-4610-a37c-27382e50dfde service glance] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:07.765639 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaefd97-0969-4610-a37c-27382e50dfde service glance] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:07.765918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaefd97-0969-4610-a37c-27382e50dfde service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:07.774826 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaefd97-0969-4610-a37c-27382e50dfde service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:07.776209 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaefd97-0969-4610-a37c-27382e50dfde service glance] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:07.784898 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1429/2838] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:03:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 22:03:23.150881 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98613c-4e9b-44db-9a84-de62b3dbe2f1 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:23.211087 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98613c-4e9b-44db-9a84-de62b3dbe2f1 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:23.215609 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9c271-e252-4b35-a1c5-1b4f876f1468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:23.216468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9c271-e252-4b35-a1c5-1b4f876f1468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:23.216840 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9c271-e252-4b35-a1c5-1b4f876f1468 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:23.217122 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9c271-e252-4b35-a1c5-1b4f876f1468 admin admin] PATH_INFO: `/v3/users/d009c3d45dba4e7284a7cab36f785263` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:23.218308 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9c271-e252-4b35-a1c5-1b4f876f1468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d009c3d45dba4e7284a7cab36f785263)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:23.220307 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9c271-e252-4b35-a1c5-1b4f876f1468 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:23.264198 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d0d9c271-e252-4b35-a1c5-1b4f876f1468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd009c3d45dba4e7284a7cab36f785263'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:23.276094 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d0d9c271-e252-4b35-a1c5-1b4f876f1468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd009c3d45dba4e7284a7cab36f785263'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:23.276759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d0d9c271-e252-4b35-a1c5-1b4f876f1468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd009c3d45dba4e7284a7cab36f785263'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:23.277399 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d0d9c271-e252-4b35-a1c5-1b4f876f1468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd009c3d45dba4e7284a7cab36f785263'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:23.283974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-d0d9c271-e252-4b35-a1c5-1b4f876f1468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd009c3d45dba4e7284a7cab36f785263'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:23.295911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1410/2839] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:03:23 2022] DELETE /identity/v3/users/d009c3d45dba4e7284a7cab36f785263 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:03:23.300355 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568ecf-0405-42bc-8b12-fdb3605a9c38 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:23.300846 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568ecf-0405-42bc-8b12-fdb3605a9c38 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:23.300961 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568ecf-0405-42bc-8b12-fdb3605a9c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:23.317426 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-d9568ecf-0405-42bc-8b12-fdb3605a9c38 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:03:23.318655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d9568ecf-0405-42bc-8b12-fdb3605a9c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:23.374338 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1430/2840] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:03:23.384464 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9c271-e252-4b35-a1c5-1b4f876f1468 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:23.385813 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-d0d9c271-e252-4b35-a1c5-1b4f876f1468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:23.441321 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9c271-e252-4b35-a1c5-1b4f876f1468 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:23.445607 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a87fc-ad22-426d-b9b5-ad9c5a4ce9ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:23.446247 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a87fc-ad22-426d-b9b5-ad9c5a4ce9ab service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:23.446537 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a87fc-ad22-426d-b9b5-ad9c5a4ce9ab service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:23.446852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a87fc-ad22-426d-b9b5-ad9c5a4ce9ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:23.455746 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a87fc-ad22-426d-b9b5-ad9c5a4ce9ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:23.457209 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a87fc-ad22-426d-b9b5-ad9c5a4ce9ab service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:23.466489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1411/2841] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:03:23.723825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9568ecf-0405-42bc-8b12-fdb3605a9c38 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:23.725300 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-d9568ecf-0405-42bc-8b12-fdb3605a9c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:23.739703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9568ecf-0405-42bc-8b12-fdb3605a9c38 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:23.745742 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c875f1-68f9-4231-93ff-9ad419056894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:23.746666 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c875f1-68f9-4231-93ff-9ad419056894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:23.746912 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c875f1-68f9-4231-93ff-9ad419056894 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:23.747193 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c875f1-68f9-4231-93ff-9ad419056894 admin admin] PATH_INFO: `/v3/projects/3faebb433d194e88931d6374768b9b9c` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:23.748394 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c875f1-68f9-4231-93ff-9ad419056894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3faebb433d194e88931d6374768b9b9c)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:23.750288 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c875f1-68f9-4231-93ff-9ad419056894 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:23.795875 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-70c875f1-68f9-4231-93ff-9ad419056894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3faebb433d194e88931d6374768b9b9c', 'request_id': 'req-70c875f1-68f9-4231-93ff-9ad419056894'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:23.801993 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-70c875f1-68f9-4231-93ff-9ad419056894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3faebb433d194e88931d6374768b9b9c', 'request_id': 'req-70c875f1-68f9-4231-93ff-9ad419056894'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:23.809576 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-70c875f1-68f9-4231-93ff-9ad419056894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3faebb433d194e88931d6374768b9b9c', 'request_id': 'req-70c875f1-68f9-4231-93ff-9ad419056894'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:23.819994 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-70c875f1-68f9-4231-93ff-9ad419056894 admin admin] The token cache is being invalidate because project 3faebb433d194e88931d6374768b9b9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:03:23.820287 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-70c875f1-68f9-4231-93ff-9ad419056894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:23.820925 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-70c875f1-68f9-4231-93ff-9ad419056894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:03:23.822462 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1431/2842] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:03:23 2022] DELETE /identity/v3/projects/3faebb433d194e88931d6374768b9b9c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 22:03:23.828269 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af2189-6fa8-4f84-9eb4-f6b87af9be81 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:23.828585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af2189-6fa8-4f84-9eb4-f6b87af9be81 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:23.828863 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af2189-6fa8-4f84-9eb4-f6b87af9be81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:23.845912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-65af2189-6fa8-4f84-9eb4-f6b87af9be81 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:03:23.847219 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-65af2189-6fa8-4f84-9eb4-f6b87af9be81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:23.905724 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1412/2843] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:03:23.913962 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808284-ef42-4937-8869-d8714f3da3a7 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:23.914388 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808284-ef42-4937-8869-d8714f3da3a7 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:23.914681 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808284-ef42-4937-8869-d8714f3da3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:23.933258 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-27808284-ef42-4937-8869-d8714f3da3a7 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:03:23.934465 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-27808284-ef42-4937-8869-d8714f3da3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:23.942402 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1432/2844] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:03:23.946737 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af2189-6fa8-4f84-9eb4-f6b87af9be81 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:23.956285 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af2189-6fa8-4f84-9eb4-f6b87af9be81 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:23.962173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfee09-ae9d-4ac1-a9bd-9ec6dc1e9e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:23.963468 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfee09-ae9d-4ac1-a9bd-9ec6dc1e9e31 admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:23.963899 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfee09-ae9d-4ac1-a9bd-9ec6dc1e9e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:23.964131 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfee09-ae9d-4ac1-a9bd-9ec6dc1e9e31 admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:23.965007 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfee09-ae9d-4ac1-a9bd-9ec6dc1e9e31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 22:03:23.965377 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfee09-ae9d-4ac1-a9bd-9ec6dc1e9e31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:23.967252 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfee09-ae9d-4ac1-a9bd-9ec6dc1e9e31 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:23.980338 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1413/2845] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 22:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 22:03:23.984610 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27808284-ef42-4937-8869-d8714f3da3a7 None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:23.994839 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27808284-ef42-4937-8869-d8714f3da3a7 None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:23.999554 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1450344-36d9-48f4-b384-cc5986433462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:24.000208 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1450344-36d9-48f4-b384-cc5986433462 admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:24.000494 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1450344-36d9-48f4-b384-cc5986433462 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:24.000770 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1450344-36d9-48f4-b384-cc5986433462 admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:24.001478 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1450344-36d9-48f4-b384-cc5986433462 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:24.003421 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1450344-36d9-48f4-b384-cc5986433462 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:24.027367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1433/2846] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:03:23 2022] POST /identity/v3/projects => generated 379 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:03:24.031445 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfee09-ae9d-4ac1-a9bd-9ec6dc1e9e31 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:24.041859 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfee09-ae9d-4ac1-a9bd-9ec6dc1e9e31 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:24.094764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21714df2-c617-4e5d-b022-1d7741a59119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:24.095419 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21714df2-c617-4e5d-b022-1d7741a59119 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:24.095702 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21714df2-c617-4e5d-b022-1d7741a59119 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:24.095982 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21714df2-c617-4e5d-b022-1d7741a59119 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:24.096659 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21714df2-c617-4e5d-b022-1d7741a59119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:24.098585 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21714df2-c617-4e5d-b022-1d7741a59119 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:24.117606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1414/2847] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:03:24 2022] POST /identity/v3/users => generated 290 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:03:24.121577 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1450344-36d9-48f4-b384-cc5986433462 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:24.131547 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1450344-36d9-48f4-b384-cc5986433462 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:24.136187 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a9c1c-995b-4339-99e7-5ed42eb30c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:24.136820 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a9c1c-995b-4339-99e7-5ed42eb30c79 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:24.137100 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a9c1c-995b-4339-99e7-5ed42eb30c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:24.137381 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a9c1c-995b-4339-99e7-5ed42eb30c79 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:24.138093 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a9c1c-995b-4339-99e7-5ed42eb30c79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:24.139703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a9c1c-995b-4339-99e7-5ed42eb30c79 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:24.150121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1434/2848] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 22:03:24.153771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21714df2-c617-4e5d-b022-1d7741a59119 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:24.164298 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21714df2-c617-4e5d-b022-1d7741a59119 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:24.168809 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba981b1f-c5d2-41c3-9086-ad5c198a253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:24.169454 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba981b1f-c5d2-41c3-9086-ad5c198a253b admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:24.169732 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba981b1f-c5d2-41c3-9086-ad5c198a253b admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:24.170009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba981b1f-c5d2-41c3-9086-ad5c198a253b admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:24.170757 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba981b1f-c5d2-41c3-9086-ad5c198a253b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:24.172339 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba981b1f-c5d2-41c3-9086-ad5c198a253b admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:24.184693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1415/2849] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:03:24.189561 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a9c1c-995b-4339-99e7-5ed42eb30c79 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:24.199537 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a9c1c-995b-4339-99e7-5ed42eb30c79 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:24.204136 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a1d0c-232f-4228-80cd-0d9a0625e03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:24.204996 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a1d0c-232f-4228-80cd-0d9a0625e03e admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:24.205282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a1d0c-232f-4228-80cd-0d9a0625e03e admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:24.205559 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a1d0c-232f-4228-80cd-0d9a0625e03e admin admin] PATH_INFO: `/v3/projects/2cbf8c168c0549d89ee2ab4f8adf6f34/users/9f0a110ecee84825bffd380d8c664ed4/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:24.219918 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a1d0c-232f-4228-80cd-0d9a0625e03e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cbf8c168c0549d89ee2ab4f8adf6f34, user_id=9f0a110ecee84825bffd380d8c664ed4, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:24.222603 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a1d0c-232f-4228-80cd-0d9a0625e03e admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:24.235560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1435/2850] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:03:24 2022] PUT /identity/v3/projects/2cbf8c168c0549d89ee2ab4f8adf6f34/users/9f0a110ecee84825bffd380d8c664ed4/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:03:24.240634 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565c471-165c-42c3-b20f-95104b0a4b8d None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:24.240928 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565c471-165c-42c3-b20f-95104b0a4b8d None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:24.241230 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565c471-165c-42c3-b20f-95104b0a4b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:24.257867 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-1565c471-165c-42c3-b20f-95104b0a4b8d None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:03:24.259094 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-1565c471-165c-42c3-b20f-95104b0a4b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:24.314154 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1416/2851] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:03:24.323519 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a1d0c-232f-4228-80cd-0d9a0625e03e admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:24.325253 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e02a1d0c-232f-4228-80cd-0d9a0625e03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:24.381522 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a1d0c-232f-4228-80cd-0d9a0625e03e admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:24.385167 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d87b13-0e46-4f57-a9b1-aed11ab82844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:24.385803 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d87b13-0e46-4f57-a9b1-aed11ab82844 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:24.386083 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d87b13-0e46-4f57-a9b1-aed11ab82844 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:24.386365 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d87b13-0e46-4f57-a9b1-aed11ab82844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:24.395101 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d87b13-0e46-4f57-a9b1-aed11ab82844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:24.396532 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d87b13-0e46-4f57-a9b1-aed11ab82844 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:24.406192 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1436/2852] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:03:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:03:24.994214 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ad72f-c7db-41b3-8e38-b7ae127b786f None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:24.994557 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ad72f-c7db-41b3-8e38-b7ae127b786f None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:24.994885 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ad72f-c7db-41b3-8e38-b7ae127b786f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:25.020765 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-975ad72f-c7db-41b3-8e38-b7ae127b786f None None] MFA Rules not processed for user `9f0a110ecee84825bffd380d8c664ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:03:25.021993 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-975ad72f-c7db-41b3-8e38-b7ae127b786f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:25.084384 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1417/2853] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:03:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:03:25.096231 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d87b13-0e46-4f57-a9b1-aed11ab82844 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:25.097626 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-14d87b13-0e46-4f57-a9b1-aed11ab82844 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:25.153894 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d87b13-0e46-4f57-a9b1-aed11ab82844 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:25.157587 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d876d3-d659-43e8-82f1-a5f5cba3dfb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86c4a81ce97b4fd78889fae6c36bf6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:25.158238 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d876d3-d659-43e8-82f1-a5f5cba3dfb1 service cinder] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:25.158518 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d876d3-d659-43e8-82f1-a5f5cba3dfb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:25.158819 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d876d3-d659-43e8-82f1-a5f5cba3dfb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:25.168737 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d876d3-d659-43e8-82f1-a5f5cba3dfb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:25.170174 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d876d3-d659-43e8-82f1-a5f5cba3dfb1 service cinder] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:25.178887 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1437/2854] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:03:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:03:26.499670 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c5e86-589a-42fb-bf6e-a2fe56cdb9e0 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:26.500151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c5e86-589a-42fb-bf6e-a2fe56cdb9e0 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:26.500270 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c5e86-589a-42fb-bf6e-a2fe56cdb9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:26.516995 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-1e6c5e86-589a-42fb-bf6e-a2fe56cdb9e0 None None] MFA Rules not processed for user `9f0a110ecee84825bffd380d8c664ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:03:26.518177 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-1e6c5e86-589a-42fb-bf6e-a2fe56cdb9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:26.525414 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1418/2855] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:03:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:03:26.754172 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d876d3-d659-43e8-82f1-a5f5cba3dfb1 service cinder] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:26.763324 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d876d3-d659-43e8-82f1-a5f5cba3dfb1 service cinder] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:26.766858 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fc686-d7ad-4d77-a4e5-90a2130c33c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:26.767476 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fc686-d7ad-4d77-a4e5-90a2130c33c2 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:26.767753 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fc686-d7ad-4d77-a4e5-90a2130c33c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:26.768024 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fc686-d7ad-4d77-a4e5-90a2130c33c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:26.769745 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-730fc686-d7ad-4d77-a4e5-90a2130c33c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:26.779576 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fc686-d7ad-4d77-a4e5-90a2130c33c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:26.780995 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fc686-d7ad-4d77-a4e5-90a2130c33c2 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:26.789355 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1438/2856] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:03:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:03:28.110753 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c5e86-589a-42fb-bf6e-a2fe56cdb9e0 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:28.112145 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-1e6c5e86-589a-42fb-bf6e-a2fe56cdb9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:28.178783 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c5e86-589a-42fb-bf6e-a2fe56cdb9e0 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:28.183771 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fb98c-2015-4d3b-8e44-39dca73bee0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8124ab8abcc640c5a48b3e0216afa48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:28.184425 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fb98c-2015-4d3b-8e44-39dca73bee0d service placement] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:28.184709 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fb98c-2015-4d3b-8e44-39dca73bee0d service placement] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:28.184984 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fb98c-2015-4d3b-8e44-39dca73bee0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:28.186811 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-a54fb98c-2015-4d3b-8e44-39dca73bee0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:28.199347 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fb98c-2015-4d3b-8e44-39dca73bee0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:28.200957 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fb98c-2015-4d3b-8e44-39dca73bee0d service placement] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:28.211345 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1419/2857] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:03:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:03:39.771740 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fc686-d7ad-4d77-a4e5-90a2130c33c2 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:03:39.773920 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-730fc686-d7ad-4d77-a4e5-90a2130c33c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:39.848823 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fc686-d7ad-4d77-a4e5-90a2130c33c2 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:03:39.854342 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbc7ba-9eda-451e-9eb2-f7a1f9ff4bfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:03:39.855275 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbc7ba-9eda-451e-9eb2-f7a1f9ff4bfc service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:03:39.855557 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbc7ba-9eda-451e-9eb2-f7a1f9ff4bfc service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:03:39.855830 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbc7ba-9eda-451e-9eb2-f7a1f9ff4bfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:03:39.857563 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-4fbbc7ba-9eda-451e-9eb2-f7a1f9ff4bfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:03:39.869144 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbc7ba-9eda-451e-9eb2-f7a1f9ff4bfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:03:39.870866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbc7ba-9eda-451e-9eb2-f7a1f9ff4bfc service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:03:39.881413 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1439/2858] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:03:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:04:30.527037 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462b058-597d-43b4-aa77-01318d7db196 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:04:30.527588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462b058-597d-43b4-aa77-01318d7db196 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:04:30.527588 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462b058-597d-43b4-aa77-01318d7db196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:04:30.553197 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-f462b058-597d-43b4-aa77-01318d7db196 None None] MFA Rules not processed for user `9f0a110ecee84825bffd380d8c664ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:04:30.554383 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f462b058-597d-43b4-aa77-01318d7db196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:04:30.577478 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1420/2859] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:04:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:04:30.798950 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbc7ba-9eda-451e-9eb2-f7a1f9ff4bfc service nova] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:04:30.835094 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbc7ba-9eda-451e-9eb2-f7a1f9ff4bfc service nova] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:04:30.838825 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac932d3f-96ea-4260-ba97-68e65e0689b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:04:30.839468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac932d3f-96ea-4260-ba97-68e65e0689b1 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:04:30.839751 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac932d3f-96ea-4260-ba97-68e65e0689b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:04:30.840026 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac932d3f-96ea-4260-ba97-68e65e0689b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:04:30.841789 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ac932d3f-96ea-4260-ba97-68e65e0689b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:04:30.863193 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac932d3f-96ea-4260-ba97-68e65e0689b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:04:30.865531 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac932d3f-96ea-4260-ba97-68e65e0689b1 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:04:30.874039 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1440/2860] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:04:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:05:25.473962 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462b058-597d-43b4-aa77-01318d7db196 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:25.512677 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462b058-597d-43b4-aa77-01318d7db196 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:25.517242 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8afe5-f14d-4ebd-94b9-3ca864191fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:25.518103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8afe5-f14d-4ebd-94b9-3ca864191fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:25.518388 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8afe5-f14d-4ebd-94b9-3ca864191fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:25.518668 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8afe5-f14d-4ebd-94b9-3ca864191fa7 admin admin] PATH_INFO: `/v3/users/9f0a110ecee84825bffd380d8c664ed4` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:25.519880 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8afe5-f14d-4ebd-94b9-3ca864191fa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f0a110ecee84825bffd380d8c664ed4)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:25.521806 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8afe5-f14d-4ebd-94b9-3ca864191fa7 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:25.567405 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-03b8afe5-f14d-4ebd-94b9-3ca864191fa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f0a110ecee84825bffd380d8c664ed4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:05:25.578458 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-03b8afe5-f14d-4ebd-94b9-3ca864191fa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f0a110ecee84825bffd380d8c664ed4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:05:25.579223 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-03b8afe5-f14d-4ebd-94b9-3ca864191fa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f0a110ecee84825bffd380d8c664ed4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:05:25.579837 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-03b8afe5-f14d-4ebd-94b9-3ca864191fa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f0a110ecee84825bffd380d8c664ed4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:05:25.587164 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-03b8afe5-f14d-4ebd-94b9-3ca864191fa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f0a110ecee84825bffd380d8c664ed4'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:05:25.599494 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1421/2861] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:05:25 2022] DELETE /identity/v3/users/9f0a110ecee84825bffd380d8c664ed4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:05:25.765925 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac932d3f-96ea-4260-ba97-68e65e0689b1 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:25.782162 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac932d3f-96ea-4260-ba97-68e65e0689b1 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:25.836277 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020289a-5a8c-4cba-a960-e3b7a3712b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:25.837661 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020289a-5a8c-4cba-a960-e3b7a3712b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:25.837939 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020289a-5a8c-4cba-a960-e3b7a3712b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:25.838216 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020289a-5a8c-4cba-a960-e3b7a3712b06 admin admin] PATH_INFO: `/v3/projects/2cbf8c168c0549d89ee2ab4f8adf6f34` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:25.839397 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020289a-5a8c-4cba-a960-e3b7a3712b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cbf8c168c0549d89ee2ab4f8adf6f34)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:25.841290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020289a-5a8c-4cba-a960-e3b7a3712b06 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:25.884052 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0020289a-5a8c-4cba-a960-e3b7a3712b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cbf8c168c0549d89ee2ab4f8adf6f34', 'request_id': 'req-0020289a-5a8c-4cba-a960-e3b7a3712b06'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:05:25.889746 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0020289a-5a8c-4cba-a960-e3b7a3712b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cbf8c168c0549d89ee2ab4f8adf6f34', 'request_id': 'req-0020289a-5a8c-4cba-a960-e3b7a3712b06'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:05:25.896662 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0020289a-5a8c-4cba-a960-e3b7a3712b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cbf8c168c0549d89ee2ab4f8adf6f34', 'request_id': 'req-0020289a-5a8c-4cba-a960-e3b7a3712b06'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:05:25.918947 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0020289a-5a8c-4cba-a960-e3b7a3712b06 admin admin] The token cache is being invalidate because project 2cbf8c168c0549d89ee2ab4f8adf6f34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:05:25.919237 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0020289a-5a8c-4cba-a960-e3b7a3712b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:05:25.919861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-0020289a-5a8c-4cba-a960-e3b7a3712b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:05:25.921428 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1441/2862] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:05:25 2022] DELETE /identity/v3/projects/2cbf8c168c0549d89ee2ab4f8adf6f34 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:05:25.926448 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0310-d8be-48f7-b654-3e99e5def7d3 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:25.926768 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0310-d8be-48f7-b654-3e99e5def7d3 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:25.927047 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0310-d8be-48f7-b654-3e99e5def7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:25.981357 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-599a0310-d8be-48f7-b654-3e99e5def7d3 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:05:25.982569 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-599a0310-d8be-48f7-b654-3e99e5def7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:05:26.040282 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1422/2863] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:05:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:05:26.048155 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5c7c3-aa03-4adf-9319-9ac7a4f0ebdf None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:26.048447 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5c7c3-aa03-4adf-9319-9ac7a4f0ebdf None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:26.048727 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5c7c3-aa03-4adf-9319-9ac7a4f0ebdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:26.066231 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-e3e5c7c3-aa03-4adf-9319-9ac7a4f0ebdf None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:05:26.067490 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-e3e5c7c3-aa03-4adf-9319-9ac7a4f0ebdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:05:26.075877 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1442/2864] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:05:26.079802 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a0310-d8be-48f7-b654-3e99e5def7d3 None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:26.089231 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a0310-d8be-48f7-b654-3e99e5def7d3 None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:26.093673 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c750a1-08a7-47cf-9ba6-c5aea023f33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:26.094311 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c750a1-08a7-47cf-9ba6-c5aea023f33f admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:26.094590 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c750a1-08a7-47cf-9ba6-c5aea023f33f admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:26.094897 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c750a1-08a7-47cf-9ba6-c5aea023f33f admin admin] PATH_INFO: `/v3/domains` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:26.095617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c750a1-08a7-47cf-9ba6-c5aea023f33f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 22:05:26.095921 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c750a1-08a7-47cf-9ba6-c5aea023f33f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:26.097513 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c750a1-08a7-47cf-9ba6-c5aea023f33f admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:26.108852 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1423/2865] 10.222.0.35 () {58 vars in 1152 bytes} [Fri May 13 22:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 22:05:26.112703 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5c7c3-aa03-4adf-9319-9ac7a4f0ebdf None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:26.122164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5c7c3-aa03-4adf-9319-9ac7a4f0ebdf None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:26.126802 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef823dd-25b6-4e16-822d-52c1a7666a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:26.127438 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef823dd-25b6-4e16-822d-52c1a7666a8a admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:26.127720 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef823dd-25b6-4e16-822d-52c1a7666a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:26.127992 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef823dd-25b6-4e16-822d-52c1a7666a8a admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:26.128675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef823dd-25b6-4e16-822d-52c1a7666a8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:26.130560 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef823dd-25b6-4e16-822d-52c1a7666a8a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:26.155712 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1443/2866] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:05:26 2022] POST /identity/v3/projects => generated 371 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 22:05:26.159890 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c750a1-08a7-47cf-9ba6-c5aea023f33f admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:26.169737 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c750a1-08a7-47cf-9ba6-c5aea023f33f admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:26.225156 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f6fdc-a4f4-491b-b9a3-de6f595d2aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:26.225815 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f6fdc-a4f4-491b-b9a3-de6f595d2aca admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:26.226099 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f6fdc-a4f4-491b-b9a3-de6f595d2aca admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:26.226375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f6fdc-a4f4-491b-b9a3-de6f595d2aca admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:26.227119 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f6fdc-a4f4-491b-b9a3-de6f595d2aca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:26.229024 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f6fdc-a4f4-491b-b9a3-de6f595d2aca admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:26.248046 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1424/2867] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:05:26 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:05:26.252146 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef823dd-25b6-4e16-822d-52c1a7666a8a admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:26.262765 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef823dd-25b6-4e16-822d-52c1a7666a8a admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:26.267363 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b4164-cd9a-48a9-9acb-4554a1606977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:26.268007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b4164-cd9a-48a9-9acb-4554a1606977 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:26.268290 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b4164-cd9a-48a9-9acb-4554a1606977 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:26.268565 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b4164-cd9a-48a9-9acb-4554a1606977 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:26.269282 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b4164-cd9a-48a9-9acb-4554a1606977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:26.270904 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b4164-cd9a-48a9-9acb-4554a1606977 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:26.280496 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1444/2868] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 22:05:26.284146 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f6fdc-a4f4-491b-b9a3-de6f595d2aca admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:26.293714 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f6fdc-a4f4-491b-b9a3-de6f595d2aca admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:26.298236 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5420f34-c895-46e2-8454-2f5c2a1d205a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:26.298904 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5420f34-c895-46e2-8454-2f5c2a1d205a admin admin] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:26.299191 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5420f34-c895-46e2-8454-2f5c2a1d205a admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:26.299470 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5420f34-c895-46e2-8454-2f5c2a1d205a admin admin] PATH_INFO: `/v3/roles` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:26.300186 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5420f34-c895-46e2-8454-2f5c2a1d205a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:26.301762 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5420f34-c895-46e2-8454-2f5c2a1d205a admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:26.311996 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1425/2869] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:05:26.316034 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b4164-cd9a-48a9-9acb-4554a1606977 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:26.326346 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b4164-cd9a-48a9-9acb-4554a1606977 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:26.330926 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84956be6-337d-498b-a687-1d25d8394e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:26.331780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84956be6-337d-498b-a687-1d25d8394e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:26.332064 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84956be6-337d-498b-a687-1d25d8394e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:26.332341 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84956be6-337d-498b-a687-1d25d8394e66 admin admin] PATH_INFO: `/v3/projects/e1c01b24fcbc4cbfa59373f7c804781d/users/b1e4fe6d0ffd4bc7bdf1314b0379e4cc/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:26.346372 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84956be6-337d-498b-a687-1d25d8394e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c01b24fcbc4cbfa59373f7c804781d, user_id=b1e4fe6d0ffd4bc7bdf1314b0379e4cc, role_id=fafcd4ed46b14e3cb8ef0d8d8f8b631d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:26.349091 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84956be6-337d-498b-a687-1d25d8394e66 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:26.362309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1445/2870] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:05:26 2022] PUT /identity/v3/projects/e1c01b24fcbc4cbfa59373f7c804781d/users/b1e4fe6d0ffd4bc7bdf1314b0379e4cc/roles/fafcd4ed46b14e3cb8ef0d8d8f8b631d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 22:05:26.367375 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc1853-bf8c-4703-9546-a12d7ec37a65 None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:26.368241 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc1853-bf8c-4703-9546-a12d7ec37a65 None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:26.368524 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc1853-bf8c-4703-9546-a12d7ec37a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:26.388489 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-5cbc1853-bf8c-4703-9546-a12d7ec37a65 None None] MFA Rules not processed for user `3b8d43206c7f4426b0be3e6d78690460`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:05:26.389706 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-5cbc1853-bf8c-4703-9546-a12d7ec37a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:05:26.445170 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1426/2871] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 22:05:26.454593 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84956be6-337d-498b-a687-1d25d8394e66 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:26.456316 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-84956be6-337d-498b-a687-1d25d8394e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:05:26.510821 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84956be6-337d-498b-a687-1d25d8394e66 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:26.514575 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac968f-82c3-4d60-af82-1db3806e6d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:26.515245 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac968f-82c3-4d60-af82-1db3806e6d82 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:26.515527 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac968f-82c3-4d60-af82-1db3806e6d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:26.515800 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac968f-82c3-4d60-af82-1db3806e6d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:26.524638 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac968f-82c3-4d60-af82-1db3806e6d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:26.526065 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac968f-82c3-4d60-af82-1db3806e6d82 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:26.535164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1446/2872] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:05:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 22:05:30.193150 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a96836-f8ee-4393-9bf7-ae3866e1efdb None None] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:30.193666 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a96836-f8ee-4393-9bf7-ae3866e1efdb None None] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:30.193770 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a96836-f8ee-4393-9bf7-ae3866e1efdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:30.221545 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.auth.core [None req-50a96836-f8ee-4393-9bf7-ae3866e1efdb None None] MFA Rules not processed for user `b1e4fe6d0ffd4bc7bdf1314b0379e4cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:05:30.222688 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-50a96836-f8ee-4393-9bf7-ae3866e1efdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:05:30.285914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1427/2873] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:05:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:05:30.290273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac968f-82c3-4d60-af82-1db3806e6d82 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:30.299556 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac968f-82c3-4d60-af82-1db3806e6d82 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:30.304095 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37954f88-2b15-4cc7-a98c-e4db2eeb0f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:30.304732 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37954f88-2b15-4cc7-a98c-e4db2eeb0f4a admin admin] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:30.305038 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37954f88-2b15-4cc7-a98c-e4db2eeb0f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:30.305311 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37954f88-2b15-4cc7-a98c-e4db2eeb0f4a admin admin] PATH_INFO: `/v3/projects` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:30.305988 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37954f88-2b15-4cc7-a98c-e4db2eeb0f4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:30.307915 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37954f88-2b15-4cc7-a98c-e4db2eeb0f4a admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:30.333835 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1447/2874] 10.222.0.35 () {60 vars in 1152 bytes} [Fri May 13 22:05:30 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 22:05:30.338261 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a96836-f8ee-4393-9bf7-ae3866e1efdb None None] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:30.348992 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a96836-f8ee-4393-9bf7-ae3866e1efdb None None] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:30.404938 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04750e7-14c8-4812-a026-902ef0bc5772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:30.405617 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04750e7-14c8-4812-a026-902ef0bc5772 admin admin] REQUEST_METHOD: `POST` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:30.405905 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04750e7-14c8-4812-a026-902ef0bc5772 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:30.406186 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04750e7-14c8-4812-a026-902ef0bc5772 admin admin] PATH_INFO: `/v3/users` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:30.406912 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04750e7-14c8-4812-a026-902ef0bc5772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:30.408848 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04750e7-14c8-4812-a026-902ef0bc5772 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:30.430199 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1428/2875] 10.222.0.35 () {60 vars in 1143 bytes} [Fri May 13 22:05:30 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 22:05:30.436656 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37954f88-2b15-4cc7-a98c-e4db2eeb0f4a admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:30.447914 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37954f88-2b15-4cc7-a98c-e4db2eeb0f4a admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:30.452667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64214ff3-d6b2-4612-8e73-6b862d822857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:30.453371 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64214ff3-d6b2-4612-8e73-6b862d822857 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:30.453651 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64214ff3-d6b2-4612-8e73-6b862d822857 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:30.453922 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64214ff3-d6b2-4612-8e73-6b862d822857 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:30.454675 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64214ff3-d6b2-4612-8e73-6b862d822857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:30.456309 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64214ff3-d6b2-4612-8e73-6b862d822857 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:30.468257 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1448/2876] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:05:30.472438 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04750e7-14c8-4812-a026-902ef0bc5772 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:30.482988 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04750e7-14c8-4812-a026-902ef0bc5772 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:30.487854 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f563d-e381-4ed2-ae42-94f12e303783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:30.488745 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f563d-e381-4ed2-ae42-94f12e303783 admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:30.489060 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f563d-e381-4ed2-ae42-94f12e303783 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:30.489341 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f563d-e381-4ed2-ae42-94f12e303783 admin admin] PATH_INFO: `/v3/projects/df34612e0a824cae9f0007f17a378452/users/10c1ecf210b94c6da1138022c64610c0/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:30.505511 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f563d-e381-4ed2-ae42-94f12e303783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df34612e0a824cae9f0007f17a378452, user_id=10c1ecf210b94c6da1138022c64610c0, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:30.508914 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f563d-e381-4ed2-ae42-94f12e303783 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:30.522426 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1429/2877] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:05:30 2022] PUT /identity/v3/projects/df34612e0a824cae9f0007f17a378452/users/10c1ecf210b94c6da1138022c64610c0/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:05:30.526330 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64214ff3-d6b2-4612-8e73-6b862d822857 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:30.536319 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64214ff3-d6b2-4612-8e73-6b862d822857 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:30.613164 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c75009-0ee2-4cff-8749-23c010d861f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:30.613838 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c75009-0ee2-4cff-8749-23c010d861f4 admin admin] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:30.614121 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c75009-0ee2-4cff-8749-23c010d861f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:30.614524 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c75009-0ee2-4cff-8749-23c010d861f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:30.615273 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c75009-0ee2-4cff-8749-23c010d861f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:30.616882 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c75009-0ee2-4cff-8749-23c010d861f4 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:30.626780 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1449/2878] 10.222.0.35 () {58 vars in 1121 bytes} [Fri May 13 22:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 22:05:30.630981 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f563d-e381-4ed2-ae42-94f12e303783 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:30.645069 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f563d-e381-4ed2-ae42-94f12e303783 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:30.650474 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd4cf4-0eb5-44e5-ac45-c4bc4f7c8eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:30.651397 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd4cf4-0eb5-44e5-ac45-c4bc4f7c8eca admin admin] REQUEST_METHOD: `PUT` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:30.651685 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd4cf4-0eb5-44e5-ac45-c4bc4f7c8eca admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:30.651967 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd4cf4-0eb5-44e5-ac45-c4bc4f7c8eca admin admin] PATH_INFO: `/v3/projects/df34612e0a824cae9f0007f17a378452/users/10c1ecf210b94c6da1138022c64610c0/roles/617e4d9732cf429b9522fc0b680ef0be` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:30.653996 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd4cf4-0eb5-44e5-ac45-c4bc4f7c8eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=df34612e0a824cae9f0007f17a378452, user_id=10c1ecf210b94c6da1138022c64610c0, role_id=617e4d9732cf429b9522fc0b680ef0be)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:30.656752 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd4cf4-0eb5-44e5-ac45-c4bc4f7c8eca admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:30.668173 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1430/2879] 10.222.0.35 () {60 vars in 1482 bytes} [Fri May 13 22:05:30 2022] PUT /identity/v3/projects/df34612e0a824cae9f0007f17a378452/users/10c1ecf210b94c6da1138022c64610c0/roles/617e4d9732cf429b9522fc0b680ef0be => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:05:35.341832 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3f301-7015-497f-835e-2f221f63b580 None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:35.342299 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3f301-7015-497f-835e-2f221f63b580 None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:35.342408 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3f301-7015-497f-835e-2f221f63b580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:35.370372 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-7fa3f301-7015-497f-835e-2f221f63b580 None None] MFA Rules not processed for user `10c1ecf210b94c6da1138022c64610c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:05:35.371541 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-7fa3f301-7015-497f-835e-2f221f63b580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:05:35.437526 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1450/2880] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:05:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 22:05:35.451390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd4cf4-0eb5-44e5-ac45-c4bc4f7c8eca admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:35.453171 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-9dcd4cf4-0eb5-44e5-ac45-c4bc4f7c8eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:05:35.538515 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd4cf4-0eb5-44e5-ac45-c4bc4f7c8eca admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:35.544487 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f2ef1-bb06-45d4-a18b-a3791853a22f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:35.545141 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f2ef1-bb06-45d4-a18b-a3791853a22f service nova] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:35.545423 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f2ef1-bb06-45d4-a18b-a3791853a22f service nova] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:35.545703 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f2ef1-bb06-45d4-a18b-a3791853a22f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:35.600953 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f2ef1-bb06-45d4-a18b-a3791853a22f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:35.603129 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f2ef1-bb06-45d4-a18b-a3791853a22f service nova] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:35.612413 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1431/2881] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:05:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:05:37.608769 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb6d96-878c-4471-a5a5-b426675599ed None None] REQUEST_METHOD: `POST` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:37.609347 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb6d96-878c-4471-a5a5-b426675599ed None None] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:37.609347 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb6d96-878c-4471-a5a5-b426675599ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:37.627203 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.auth.core [None req-02fb6d96-878c-4471-a5a5-b426675599ed None None] MFA Rules not processed for user `b1e4fe6d0ffd4bc7bdf1314b0379e4cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 22:05:37.628412 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-02fb6d96-878c-4471-a5a5-b426675599ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:05:37.635492 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1451/2882] 10.222.0.35 () {58 vars in 957 bytes} [Fri May 13 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 22:05:52.836961 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f2ef1-bb06-45d4-a18b-a3791853a22f service nova] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:05:52.838360 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-174f2ef1-bb06-45d4-a18b-a3791853a22f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:05:52.925056 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f2ef1-bb06-45d4-a18b-a3791853a22f service nova] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:05:52.930480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e104d9-9714-4032-a5f7-61d8ebdcce8d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8124ab8abcc640c5a48b3e0216afa48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:05:52.931790 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e104d9-9714-4032-a5f7-61d8ebdcce8d service placement] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:05:52.932085 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e104d9-9714-4032-a5f7-61d8ebdcce8d service placement] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:05:52.932360 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e104d9-9714-4032-a5f7-61d8ebdcce8d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:05:52.934103 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-95e104d9-9714-4032-a5f7-61d8ebdcce8d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:05:52.945392 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e104d9-9714-4032-a5f7-61d8ebdcce8d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:05:52.947000 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e104d9-9714-4032-a5f7-61d8ebdcce8d service placement] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:05:52.956619 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1432/2883] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:05:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:06:09.289283 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb6d96-878c-4471-a5a5-b426675599ed None None] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:06:09.290724 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-02fb6d96-878c-4471-a5a5-b426675599ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:06:09.345190 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb6d96-878c-4471-a5a5-b426675599ed None None] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:06:09.348938 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c26f51-6e64-473a-96bb-686791dd6a4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:06:09.349581 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c26f51-6e64-473a-96bb-686791dd6a4b service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:06:09.349866 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c26f51-6e64-473a-96bb-686791dd6a4b service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:06:09.350147 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c26f51-6e64-473a-96bb-686791dd6a4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:06:09.359468 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c26f51-6e64-473a-96bb-686791dd6a4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:06:09.360903 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c26f51-6e64-473a-96bb-686791dd6a4b service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:06:09.370110 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1452/2884] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:06:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:08:02.356519 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e104d9-9714-4032-a5f7-61d8ebdcce8d service placement] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:08:02.365740 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e104d9-9714-4032-a5f7-61d8ebdcce8d service placement] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:08:02.369369 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5e131-4117-4360-a06b-fe274adcf1e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:08:02.370001 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e131-4117-4360-a06b-fe274adcf1e4 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:08:02.370291 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e131-4117-4360-a06b-fe274adcf1e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:08:02.371245 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e131-4117-4360-a06b-fe274adcf1e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:08:02.373009 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-3af5e131-4117-4360-a06b-fe274adcf1e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:08:02.382297 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5e131-4117-4360-a06b-fe274adcf1e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:08:02.383758 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5e131-4117-4360-a06b-fe274adcf1e4 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:08:02.391580 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1433/2885] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:08:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:08:05.892252 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c26f51-6e64-473a-96bb-686791dd6a4b service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:08:05.900757 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c26f51-6e64-473a-96bb-686791dd6a4b service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:08:05.904423 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa1fc3-b968-4e3d-8417-6ac8345e04a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:08:05.905048 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa1fc3-b968-4e3d-8417-6ac8345e04a4 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:08:05.905331 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa1fc3-b968-4e3d-8417-6ac8345e04a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:08:05.905608 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa1fc3-b968-4e3d-8417-6ac8345e04a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:08:05.907367 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-83fa1fc3-b968-4e3d-8417-6ac8345e04a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:08:05.933491 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa1fc3-b968-4e3d-8417-6ac8345e04a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:08:05.934965 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa1fc3-b968-4e3d-8417-6ac8345e04a4 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:08:05.962185 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1453/2886] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:08:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:08:07.144911 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5e131-4117-4360-a06b-fe274adcf1e4 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:08:07.153482 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5e131-4117-4360-a06b-fe274adcf1e4 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:08:07.157155 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010fe65-1411-43d3-b065-1cf8c3d6445e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:08:07.157788 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010fe65-1411-43d3-b065-1cf8c3d6445e service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:08:07.158074 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010fe65-1411-43d3-b065-1cf8c3d6445e service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:08:07.158352 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010fe65-1411-43d3-b065-1cf8c3d6445e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:08:07.160151 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-e010fe65-1411-43d3-b065-1cf8c3d6445e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:08:07.173340 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010fe65-1411-43d3-b065-1cf8c3d6445e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:08:07.174784 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010fe65-1411-43d3-b065-1cf8c3d6445e service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:08:07.186394 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1434/2887] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:08:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:08:20.134145 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa1fc3-b968-4e3d-8417-6ac8345e04a4 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:08:20.143349 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa1fc3-b968-4e3d-8417-6ac8345e04a4 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:08:20.147299 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cf7fe-624e-414a-a0fa-3de7cd66a028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:08:20.147944 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cf7fe-624e-414a-a0fa-3de7cd66a028 service neutron] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:08:20.148231 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cf7fe-624e-414a-a0fa-3de7cd66a028 service neutron] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:08:20.148576 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cf7fe-624e-414a-a0fa-3de7cd66a028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:08:20.154263 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ef5cf7fe-624e-414a-a0fa-3de7cd66a028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:08:20.164205 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5cf7fe-624e-414a-a0fa-3de7cd66a028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:08:20.165667 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5cf7fe-624e-414a-a0fa-3de7cd66a028 service neutron] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:08:20.173949 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1454/2888] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:08:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:08:20.256957 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010fe65-1411-43d3-b065-1cf8c3d6445e service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:08:20.265544 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010fe65-1411-43d3-b065-1cf8c3d6445e service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:08:20.269234 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a33c63-3a8e-4cb5-972f-73c3b5d29754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4df058606c448a1bb757020e869ac33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:08:20.269867 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a33c63-3a8e-4cb5-972f-73c3b5d29754 service neutron] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:08:20.270148 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a33c63-3a8e-4cb5-972f-73c3b5d29754 service neutron] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:08:20.270421 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a33c63-3a8e-4cb5-972f-73c3b5d29754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:08:20.272204 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-f4a33c63-3a8e-4cb5-972f-73c3b5d29754 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:08:20.285362 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a33c63-3a8e-4cb5-972f-73c3b5d29754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:08:20.286831 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a33c63-3a8e-4cb5-972f-73c3b5d29754 service neutron] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:08:20.298079 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1435/2889] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:08:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:09:46.874977 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cf7fe-624e-414a-a0fa-3de7cd66a028 service neutron] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:09:46.876458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-ef5cf7fe-624e-414a-a0fa-3de7cd66a028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:09:46.893936 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cf7fe-624e-414a-a0fa-3de7cd66a028 service neutron] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:09:46.899772 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36913d-7109-423f-a580-8e4de0c4408e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ef43c6c304bd3a27e085bf4da8a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:09:46.900435 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36913d-7109-423f-a580-8e4de0c4408e service nova] REQUEST_METHOD: `GET` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:09:46.900738 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36913d-7109-423f-a580-8e4de0c4408e service nova] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:09:46.901103 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36913d-7109-423f-a580-8e4de0c4408e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:09:46.902929 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-fe36913d-7109-423f-a580-8e4de0c4408e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:09:46.916007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36913d-7109-423f-a580-8e4de0c4408e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:09:46.917598 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36913d-7109-423f-a580-8e4de0c4408e service nova] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:09:46.931847 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1455/2890] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:09:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 22:09:57.765789 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a33c63-3a8e-4cb5-972f-73c3b5d29754 service neutron] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:09:57.829693 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a33c63-3a8e-4cb5-972f-73c3b5d29754 service neutron] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:09:57.834292 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7f48c-f921-46b6-bb22-0e754299c670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:09:57.835199 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7f48c-f921-46b6-bb22-0e754299c670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:09:57.835480 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7f48c-f921-46b6-bb22-0e754299c670 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:09:57.835759 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7f48c-f921-46b6-bb22-0e754299c670 admin admin] PATH_INFO: `/v3/users/b1e4fe6d0ffd4bc7bdf1314b0379e4cc` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:09:57.837591 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7f48c-f921-46b6-bb22-0e754299c670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1e4fe6d0ffd4bc7bdf1314b0379e4cc)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:09:57.839576 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7f48c-f921-46b6-bb22-0e754299c670 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:09:57.886868 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-fbc7f48c-f921-46b6-bb22-0e754299c670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1e4fe6d0ffd4bc7bdf1314b0379e4cc'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:57.897830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-fbc7f48c-f921-46b6-bb22-0e754299c670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1e4fe6d0ffd4bc7bdf1314b0379e4cc'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:57.898484 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-fbc7f48c-f921-46b6-bb22-0e754299c670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1e4fe6d0ffd4bc7bdf1314b0379e4cc'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:57.899764 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-fbc7f48c-f921-46b6-bb22-0e754299c670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1e4fe6d0ffd4bc7bdf1314b0379e4cc'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:57.906429 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-fbc7f48c-f921-46b6-bb22-0e754299c670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1e4fe6d0ffd4bc7bdf1314b0379e4cc'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:57.918858 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1436/2891] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:09:57 2022] DELETE /identity/v3/users/b1e4fe6d0ffd4bc7bdf1314b0379e4cc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:09:58.084814 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36913d-7109-423f-a580-8e4de0c4408e service nova] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:09:58.095405 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36913d-7109-423f-a580-8e4de0c4408e service nova] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:09:58.158063 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:09:58.158966 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:09:58.159254 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:09:58.159531 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] PATH_INFO: `/v3/projects/e1c01b24fcbc4cbfa59373f7c804781d` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:09:58.160709 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1c01b24fcbc4cbfa59373f7c804781d)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:09:58.162655 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:09:58.207094 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1c01b24fcbc4cbfa59373f7c804781d', 'request_id': 'req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.213007 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1c01b24fcbc4cbfa59373f7c804781d', 'request_id': 'req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.220231 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1c01b24fcbc4cbfa59373f7c804781d', 'request_id': 'req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.230424 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] The token cache is being invalidate because project e1c01b24fcbc4cbfa59373f7c804781d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:09:58.230777 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.231379 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.232946 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1456/2892] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:09:58 2022] DELETE /identity/v3/projects/e1c01b24fcbc4cbfa59373f7c804781d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:09:58.237004 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7f48c-f921-46b6-bb22-0e754299c670 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:09:58.238390 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-fbc7f48c-f921-46b6-bb22-0e754299c670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:09:58.251571 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7f48c-f921-46b6-bb22-0e754299c670 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:09:58.303969 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b87c6e-baa2-41b0-bbb2-942ee2d90520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:09:58.304765 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b87c6e-baa2-41b0-bbb2-942ee2d90520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:09:58.305557 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b87c6e-baa2-41b0-bbb2-942ee2d90520 admin admin] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:09:58.305830 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b87c6e-baa2-41b0-bbb2-942ee2d90520 admin admin] PATH_INFO: `/v3/users/10c1ecf210b94c6da1138022c64610c0` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:09:58.307030 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b87c6e-baa2-41b0-bbb2-942ee2d90520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10c1ecf210b94c6da1138022c64610c0)` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:09:58.308990 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b87c6e-baa2-41b0-bbb2-942ee2d90520 admin admin] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:09:58.354041 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c2b87c6e-baa2-41b0-bbb2-942ee2d90520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10c1ecf210b94c6da1138022c64610c0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.365288 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c2b87c6e-baa2-41b0-bbb2-942ee2d90520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10c1ecf210b94c6da1138022c64610c0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.365974 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c2b87c6e-baa2-41b0-bbb2-942ee2d90520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10c1ecf210b94c6da1138022c64610c0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.366606 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c2b87c6e-baa2-41b0-bbb2-942ee2d90520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10c1ecf210b94c6da1138022c64610c0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.373725 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.notifications [None req-c2b87c6e-baa2-41b0-bbb2-942ee2d90520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10c1ecf210b94c6da1138022c64610c0'} {{(pid=91431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.386505 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1437/2893] 10.222.0.35 () {58 vars in 1223 bytes} [Fri May 13 22:09:58 2022] DELETE /identity/v3/users/10c1ecf210b94c6da1138022c64610c0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:09:58.552101 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] Authenticating user token {{(pid=91430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:09:58.553529 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.fernet_utils [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:09:58.570191 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b7d4d-8a79-4ead-b52b-b4f637955cb5 admin admin] Validating token access rules against request {{(pid=91430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:09:58.626265 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796548a5-dc14-427d-8e2f-3541c80a9ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b8d43206c7f4426b0be3e6d78690460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44b04a795b974ef6b1191a457380d73c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:09:58.627151 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796548a5-dc14-427d-8e2f-3541c80a9ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:09:58.627440 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796548a5-dc14-427d-8e2f-3541c80a9ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:09:58.627719 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796548a5-dc14-427d-8e2f-3541c80a9ca1 admin admin] PATH_INFO: `/v3/projects/df34612e0a824cae9f0007f17a378452` {{(pid=91430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:09:58.628891 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796548a5-dc14-427d-8e2f-3541c80a9ca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df34612e0a824cae9f0007f17a378452)` {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:09:58.630861 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796548a5-dc14-427d-8e2f-3541c80a9ca1 admin admin] RBAC: Authorization granted {{(pid=91430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:09:58.675152 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-796548a5-dc14-427d-8e2f-3541c80a9ca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df34612e0a824cae9f0007f17a378452', 'request_id': 'req-796548a5-dc14-427d-8e2f-3541c80a9ca1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.681696 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-796548a5-dc14-427d-8e2f-3541c80a9ca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df34612e0a824cae9f0007f17a378452', 'request_id': 'req-796548a5-dc14-427d-8e2f-3541c80a9ca1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.688313 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-796548a5-dc14-427d-8e2f-3541c80a9ca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df34612e0a824cae9f0007f17a378452', 'request_id': 'req-796548a5-dc14-427d-8e2f-3541c80a9ca1'} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.698179 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-796548a5-dc14-427d-8e2f-3541c80a9ca1 admin admin] The token cache is being invalidate because project df34612e0a824cae9f0007f17a378452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 22:09:58.698458 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-796548a5-dc14-427d-8e2f-3541c80a9ca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.699085 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: DEBUG keystone.notifications [None req-796548a5-dc14-427d-8e2f-3541c80a9ca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 22:09:58.700627 dv-cinder-840516-1-iscsi devstack@keystone.service[91430]: [pid: 91430|app: 0|req: 1457/2894] 10.222.0.35 () {58 vars in 1232 bytes} [Fri May 13 22:09:58 2022] DELETE /identity/v3/projects/df34612e0a824cae9f0007f17a378452 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 22:10:57.614386 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b87c6e-baa2-41b0-bbb2-942ee2d90520 admin admin] Authenticating user token {{(pid=91431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 22:10:57.615871 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-c2b87c6e-baa2-41b0-bbb2-942ee2d90520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:10:57.689773 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b87c6e-baa2-41b0-bbb2-942ee2d90520 admin admin] Validating token access rules against request {{(pid=91431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 22:10:57.748987 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74234f0c-b112-42af-b8e8-2dc593aee69d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8124ab8abcc640c5a48b3e0216afa48d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac834e84b2c84261ade311a6dfbbeb20', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 22:10:57.749640 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74234f0c-b112-42af-b8e8-2dc593aee69d service placement] REQUEST_METHOD: `GET` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 22:10:57.749922 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74234f0c-b112-42af-b8e8-2dc593aee69d service placement] SCRIPT_NAME: `/identity` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 22:10:57.750200 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74234f0c-b112-42af-b8e8-2dc593aee69d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 22:10:57.751994 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.fernet_utils [None req-74234f0c-b112-42af-b8e8-2dc593aee69d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 22:10:57.763447 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74234f0c-b112-42af-b8e8-2dc593aee69d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 22:10:57.765002 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74234f0c-b112-42af-b8e8-2dc593aee69d service placement] RBAC: Authorization granted {{(pid=91431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 22:10:57.774601 dv-cinder-840516-1-iscsi devstack@keystone.service[91431]: [pid: 91431|app: 0|req: 1438/2895] 10.222.0.35 () {60 vars in 1365 bytes} [Fri May 13 22:10:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)